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FOREWORD 


This volume is a contribution to psycho-technology. Mr. 
Moore has made, in the following experimental and statistical 
study, the first approach toward a scientific solution of the problem 
of placement of engineering graduates in a great American indus- 
trial organization. 

It is a bold and novel undertaking, characteristic of a young 
century, to essay the application of the principles and technique 
of scientific method to the solution of human problems in industry. 
Yet it needs no rare wit to see that these very problems of human 
adjustment must hold during the twentieth century the focus of 
study and invention which during the nineteenth century were 
chiefly concentrated on the problem of perfecting the mechanics of 
manufacture. Human engineering is destined to a development 
comparable to that experienced by mechanical, chemical, and elec- 
trical engineering, when they first began to draw largely on the 
lustily growing sciences of chemistry and physics. Thanks to 
the foundations laid by these sciences, modern industry has a 
highly developed technology of materials and processes. It asks 
now for a technology of human nature. 

This study in principles and practices of personnel selection is 
not a popular treatise. But its appeal will be felt by at least three 
types of readers: the thoughtful leader of industry who ponders 
the trend of scientific experimentation on human problems; the 
young engineering graduate who is debating in his own mind 
whether to become a designer, a manager, of a salesman; and the 
psychologist who is watching, with some apprehension, the move- 
ment to put into practical use the tools he has forged. 


W. V. BincHamM, 
Division of Applied Psychology, 
Carnegie Institute of Technology. 








PREFACE 


This report is a description of a specific study on a practical 
problem. Certain principles and practices are proposed as conclu- 
sions from this study. The practical problem was to determine 
methods and means for selecting young engineers just being 
graduated from college and placing them in the type of work 
which they could do best in a large electric manufacturing com- 
pany. The outstanding feature of the problem was to select the 
men best qualified to develop into sales engineers; that is, en- 
gineers who meet the public and sell electrical machinery. The 
study was made primarily to solve the practical problem; and 
then the principles underlying the solution were formulated in 
order that they might be applied to further problems. In other 
words, the research was made for service, in the belief that it was 
possible to give service and at the same time make a contribution 
to science. 

As a practical investigation the work is already giving service. 
The results already obtained on one hundred seven engineers 
employed by the firm during the first year that the new methods 
recommended here have been used, are very similar to the results 
obtained on the group studied in this research. Assuming that 
the executives classified the men correctly at the end of a year, 
we can say that the results of the test alone would have classified 
seventy per cent of the men correctly on the day that they were 
employed. Beginning with June, 1921, the company will use the 
test and other methods recommended here as definite aids in clas- 
sifying and placing graduate engineers whom they employ. 

Progress in psychology, as in any other science, has been made 
by specialization and intensive study on a specific problem. Quite 
often in science both the motive and the opportunity has been a 
practical problem in industry. The growing realization that the 
human factor is a most important one in industry has given both 
motive and opportunities for research on specific problems in psy- 


Vv 


























vi PREFACE 


chology, the science of human behavior. The business world is ap- 
pealing to psychology as a science to solve specific problems. Some 
psychologists have already brought forth and exploited the facts, 
the technique, and the point of view of their science, in the hope 
that the science might be advanced as a science as well as be made 
to serve man in utilitarian ways. For accomplishing both pur- 
poses, the first and most important principle is that one definitely 
narrow and specific problem after another must be attacked and 
solved to contribute its portion to the cumulative content of the 
science. 

This report of a study on such a specific problem has been 
written in a comparatively brief and concise form. It is intended 
for practical use; but also it is intended to be read in its entirety 
only by those who understand the language of psychology and 
statistical methods. The original tables of data are not presented ; 
and much of the explanation is omitted which might be included 
in an exhaustive treatise. The tables and all other details of the 
study are on file with the Division of Applied Psychology, Car- 
negie Institute of Technology, and are available for all who 
may wish to examine them. However, it is believed that the 
report is sufficiently complete to enable those who have an in- 
telligent interest in such a problem to grasp what the study has 
to give. 
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PERSONNEL SELECTION OF GRADUATE 
ENGINEERS 


THE DIFFERENTIATION OF APPRENTICE ENGINEERS FOR 
TRAINING AS SALESMEN, DESIGNERS AND 
EXECUTIVES OF PRODUCTION 


PART I. THE PROBLEM 


I. THe PRACTICAL PROBLEM 


The Westinghouse Electric and Manufacturing Company 
each year employs about three hundred engineers who have just 
been graduated from technical schools and colleges. All these 
men have had practically the same training; but they must be 
differentiated and placed in different lines of engineering accord- 
ing to their special interests and aptitudes. The aim is not to 
discover which engineers are best for any particular kind of 
work; for a few very capable men might be best in all lines; 
but the aim is to discover in what line of work each engineer 
will function most efficiently and satisfactorily. That is, the 
desired end is to have each member of the organization so placed 
that he is able to render his greatest service to the Company by 
having the greatest opportunity to develop his capacities and 
promote himself in the line of work which he most enjoys. The 
practical problem is to determine methods and means for select- 
ing and placing young engineers in the type of work which they 
can do best. 

For selecting, differentiating, and properly assigning grad- 
uate engineers to the various phases of engineering and sales- 
manship, five methods or means and combinations of them are 
possible, namely : 

I. Grades made in the technical schools; 

2. The grades, try-outs, and observations secured during the 

year of preliminary training in the firm’s educational de- 
partment ; 
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2 BRUCE V. MOORE 


3. Rating scales; 

4. Interests and desires of graduate engineers ; 

5. Psychological tests. 

The second and fourth, interests, and observation during train- 
ing are the ones emphasized at present; and in fact they are the 
best methods, at least until new methods are evaluated. How- 
ever, these methods are not entirely satisfactory nor ideal; for 
the interests or choices of the men are uncertain guides and are 
not always identical with the needs of the company; and often the 
eraduate engineers cannot decide themselves what work they 
would prefer. The process of trying out under observation dur- 
ing training can be done only after a certain amount of selec- 
tion and differentiation has been made, and the Company has 
more or less committed itself in regard to its future engineers 
and salesmen. The training could be made more definite and 
intensive if it were known from the beginning, to just what line 
of work the graduate engineers should be assigned. Moreover, a 
more or less definite quota is needed for each line of work, and 
those men should be selected from the technical schools who 
would most exactly be the number required for each line of work. 
Finally, but not least important, the method of trying out under 
observation fails to reveal for what the graduate engineer is best 
fitted; and at best, there is often an uncertainty or question as 
to what work should be assigned. To meet the conditions just 
described, the first and second methods are analyzed, but atten- 
tion is concentrated upon the development and application of the 
third, fourth, and fifth methods named above, that is, rating 
scales, survey of actual interests, and psychological tests. 


II. Tue PsycHOLOGICAL PROBLEM INVOLVED 


Broadly speaking the psychological problem is one of indi- 
vidual differences relative to capacity for doing specific kinds of 
work. The previous policy was to classify and assign or place 
the engineers according to their interests, the practice being 
limited somewhat by the openings or needs in the Company and 
by the general impressions which the executives had of the young 
engineers. The big question raised was whether the men were 
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being placed where they could function most efficiently. Were 
their interests a reliable criterion of their ability? This leads to 
the fundamental psychological questions: (1) Is marked ability 
in a particular line of work only general ability or intelligence 
directed by interest? (2) Are there different kinds of intelli- 
gence? (3) Is there one thing called general intelligence, and 
additional abilities which are special and which vary independ- 
ently? (4) Is interest an indication of a special ability in a 
particular line or work? (5) Are there only special abilities, 
not necessarily correlating with a general average of these, 
which is called general intelligence? (6) Does success in a 
particular occupation depend upon purely intellectual ability, or 
are the more or less vaguely defined personal traits the more im- 
portant for success? (7) When it is learned which of these 
factors are important, how can they be measured so as to make 
a prediction or a statement of the probability of success in a 
particular job? Summarizing these questions, we state the psy- 
chological problem as follows: To determine what mental abili- 
ties and traits are of practicable use as criteria for differentiating 
graduate engineers and predicting their success in six different 
but closely related kinds of work. This problem includes the 
evaluation of old methods and means and the devising of new 
methods and means for determining and measuring these mental 
abilities and traits. It is not to be expected that we should answer 
all the questions stated above, which have already taken years 
of work by the best psychologists, and will require much more 
work; but these questions should be recognized as pertinent in 
the assumptions and conclusions of this study. 

This being a specific study with a definite problem in applied 
psychology, it does not include in its purpose and scope any his- 
torical or theoretical treatment of many general psychological 
problems which may be touched upon. Very few studies that 
have been published deal directly with this kind of problem. 
However, at the end of this report is appended a bibliography or 
list of studies which bear more or less specifically upon a prob- 
lem of this nature. 
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4 BRUCE V. MOORE 


PART II. THE LINES OF INVESTIGATION 
AND THE RESULTS 


III. OccuPaTIONAL DESCRIPTIONS AND ANALYSES 


The first important step in all personnel selection and place- 
ment is to secure adequate descriptions and analyses of the jobs 
to be filled. Then from these there must be prepared specifica- 
tions for the persons who will most nearly meet these require- 
ments. The specification is an interpretation of the work done 
on the job, in terms of capacities, skill, knowledge, and traits re- 
quired to do the work. In practice the degree of specificity to 
which these descriptions are carried varies considerably from 
firm to firm and with different lines of work. In general dis- 
cussions it is quite common to refer to the various vocations, 
which differentiate only the general lines of work typified by the 
various professions and trades. Often in employment and place- 
ment work, differentiations are made only into occupations, which 
are understood to be the specialties in the trades, such as punch 
press hand, lathe hand, die sinker, armature winder, etc. Strictly 
speaking, the job description is reserved for a description of a 
particular operation on a particular piece of material or appara- 
tus by means of a particular tool or machine. In the Westing- 
house Electric and Manufacturing Company these distinctions 
are made in the use of these terms by the Occupations and Rate 
Committee and by those in the Employment Department. 

In the Westinghouse Electric and Manufacturing Company 
the employment and placement of the engineers is done by the 
Educational Department. Each spring this department selects 
between two and three hundred engineers from the senior classes 
of the technical colleges; and after giving them a year of grad- 
uate training in the Educational Department at the central plant 
in East Pittsburgh, it places these men in the various lines of 
work in the organization. Before it was possible to choose any 
certain engineers as better fitted for certain lines of work, it was 
necessary to know what was expected of these engineers in each 
line of work. As there were not on record any descriptions or 
specifications of the work done by the engineers in the various 
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departments, this information had to be gotten first. The descrip- 
tions of the lines of work were obtained through the Educational 
Department. 

After conferences with some of the executives who could tell 
most about the various lines of work, a description and outline of 
the nature of the occupational description desired was given to the 
Educational Department. Under the supervision of the Educa- 
tional Department the occupational descriptions were prepared 
by the heads of the various sections in each of the departments. 
These were elaborated further by means of conferences with 
various executives in which questions were asked to clear up 
obscure details. Finally, in this manner, a conception of each 
line of work was obtained, which was detailed enough for work- 
ing intelligently on the problem of selection and placement. 

It would be out of place here to give an exhaustive description 
of each job or line of work in each section of each department. 
Moreover, such detailed descriptions are not necessary for un- 
derstanding the problem that is before us. The complete occu- 
pational descriptions are on file with the Division of Applied Psy- 
chology, Carnegie Institute of Technology, and are available for 
those who wish to examine them. It is sufficient here to have a 
description of the main classes of engineering work. In actual 
work upon the problem, only four classes of engineers are con- 
sidered, namely: (1) research workers and design engineers; 
(2) general or all other engineers in the engineering department ; 
(3) operating, service, and works management engineers; (4) 
sales engineers. This was found practicable because nearly all 
the engineers were graduates of technical colleges and had re- 
ceived essentially the same technical training. Moreover, this 
process of being admitted to the technical colleges and of being 
graduated from them, had been highly selective so that any dif- 
ferentiation among the men was possible only by comparatively 
fine discrimination. In fact, the one real differentiation that was 
consistently kept in mind in all this study was that between the 
engineers going into the sales department and all other engineers. 
This might be qualified or rather more definitely stated by ex- 
plaining that the design engineers were also differentiated from 
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6 BRUCE V. MOORE 


the other engineers, and being considered as representing the 
purest type of engineer, they were also considered as at the 
opposite extreme from the sales engineer when the differences 
were looked at as quantitative or linear in nature. The descrip- 
tions of these lines of work are given with particular attention to 
those lines considered to be most clearly differentiated. Com- 
bined with these descriptions are given the specifications for the 
kind of men considered best fitted for each particular line of 
work. 


1. Functional Classification of Engineers 


Within the organization, the engineers are classified according 
to the department and according to the class of product worked 
upon. Since the specifications for the kind of engineer needed 
depends upon the kind of work done, we must classify the en- 
gineeers according to the qualifications required to do the par- 
ticular kinds of work: (a) In all the specialized engineering 
departments which are responsible for the engineering work on 
different particular lines of electrical apparatus and machinery, 
thus excepting the General Engineering Department and the 
Testing Department, there must be design engineering. This 
calls for engineering of the purest and highest type. With this 
type of engineering is also classed the research worker in the 
Research Department. (b) All the other engineers of the en- 
gineering departments are to be considered as doing a second 
type of work. This work is engineering; but it is not to be con- 
sidered as calling for as great mathematical and technical ability 
and as much originality in that line as is needed by the design 
engineer. (c) The third class includes the engineers in Factory 
Management, the Service Department, and Operating. The 
work of these engineers brings them, to a larger extent, in con- 
tact with other people outside the organization and also with 
those inside the organization. Their work requires more exec- 
utive ability than the work done by the other groups of engin- 
eers. (d) The fourth class is that composed of Sales Engineers. 
These men are primarily engineers as well as salesmen; for to 
get and keep the good-will and confidence of the customers, they 
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must act as consulting engineers and give reliable technical advice 
to the customer or prospective customer. However, the success- 
ful engineer doing the work in the sales department must have 
a commercial sense and an ability to meet and successfully nego- 
tiate with other people, that differentiates him from the other 
engineers. 


IV. CRITERIA OF SUCCESS OF ENGINEERS 


Closely allied and of equal importance with a definite know- 
ledge of the jobs to be filled, are criteria of success of men in 
those jobs. Methods of selection and placement cannot be de- 
veloped or improved upon unless it is possible to know when bet- 
ter results are obtained. The best criteria are objective measures, 
such as records of production; but it is seldom possible to 
obtain facts on such objective measures that are comparable. 
The work of sales engineers and of design engineers is such 
that it cannot be measured by the product finished. Although 
records of sales may be kept, many other factors enter into the 
value of a sales engineer to the company. The professional 
consulting service which the engineer gives to the customer, and 
the good-will secured in return, cannot be measured in that way. 
Also, a design engineer may spend all his time for a year or 
more on one machine or piece of apparatus; but if he gives the 
piece of apparatus an original design or makes it in the nature of 
an invention, his work is scarcely measurable. However, it is 
necessary to make use of the best criteria available. 

The year spent in the Educational Department can be consid- 
ered as one phase of the engineer’s work. Therefore, grades and 
ratings on their success there are some criteria. Furthermore, 
the executives and instructors of that department come to know 
intimately all the student engineers during the year of training. 
To give some definite data against which tests and other ratings 
might be checked, the Educational Department was asked to 
rank the student engineers in the order of their general intel- 
ligence. General intelligence was defined for them as “the ability 
to analyze a problem, grasp the point, and deal with a new situa- 
tion.” Two men who both supervise the work of the engineers 
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and instruct them, worked together as a committee to rate the 
student engineers. They first classified the men into five groups 
according to their general intelligence, as defined. Then they 
ranked the men in each group in the order of their estimated 
intelligence. By combining the groups, with the highest rated 
group first, a ranking of all the engineers was secured. 

Further checks on the reliability of ratings, test scores, and 
these grades were possible by intercorrelations. Although none of 
these were absolute measures of success, it was reasonable to 
assume that high intercorrelations between different kinds of 
measures, such as between grades and tests, or between estimates 
and grades would show that there was a certain consistency in 
the measures that correlated highly with other measures, and 
that there was no cause to believe that they were unreliable. 


V. EVALUATION OF TECHNICAL SCHOOL GRADES 


The representatives of the Company, in selecting engineers 
graduating from the technical schools, did not consider primarily 
the grades received by the engineers in their college courses. They 
interviewed the seniors for twenty or thirty minutes, and then 
later inquired about the man’s grades in general. However, the 
grades were not considered as an absolute index of aman. Lead- 
ership and participation in extra-academic activities were con- 
sidered as important. It is not always easy to obtain the grades 
in such a way that extensive and consistent use can be made of 
them. I thought that probably they would show something of the 
technical training and ability of the men, but that many qualities 
making for success in engineering and salesmanship were prob- 
ably not to be inferred from such grades. However, a scientific 
investigation of this sort would not ignore them. Moreover, a 
study of these grades might give some light on the significance 
of grades given in the Company’s training courses. Accordingly, 
I made a study of the grades which graduate engineers received 
in the technical schools. Many difficulties were encountered in 
attempting to deal with technical school grades statistically and 
to derive reliable conclusions. 

Transmutation and Tabulation of Technical School Grades.— 
A letter explaining the nature of this study and the purpose in 
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requesting the grades was sent to over sixty different technical 
schools to secure the grades of 116 students. Transcripts of 
grades for 94 students were received. No two schools used the 
same form on which to record the grades; and there were at least 
seven distinctly different systems of grading with several varia- 
tions of each system. The grades of men from different schools 
had to be made comparable. There were not enough men from 
any one school to make it possible to transmute their grades into 
| standard deviations, so another method was adopted. The range 
of possible grades between the just passing grade and the highest 
possible grade, was learned either from the college catalogue or 
from the key on the transcript of grades. The median point on 
that scale was estimated and called the average grade. All such 
average grades were given a marking or value of 2. Grades 
above average were given a value of 3; and grades below average 
were given a value of 1. Grades below passing were given a 
value of 0; and exceptionally high grades in a subject, shown 
to be reliable by more than one highest marking on that subject, 
were given a value of 4. The method of transmutation of the 
grades can be shown by the following table which gives some 
comparable values. 















































TABLE I. “¢ 
Transmutation of Technical School Grades into Common Values. i 
Common Values into | ‘| 
which Grades are it 
Transmuted. re) | I 2 3 4 i 
System using per cents.. -69 | 70-79 80-89 90-97 98-100 | a. 
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System using letters..... E | D ts B A . 
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System using points.... 0-.9 | 1.0-1.9 2.0-2.9 3.0-3.7 3.8-4.0 
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In transmuting grades from institutions with high standards 
and strict grading systems, the evaluations were made higher 
than with institutions known to be below the average standing in 
the requirements it made of its students. Altogether, the trans- 
mutation was coarse as a statistical method; but it was the best 
that could be done with the data available; and it is considered 
sufficiently reliable to give significant indications. 

All the subjects studied by the engineers were classified under 
five headings or general subjects, namely,—Languages, ‘including 
English and foreign languages; Mathematics and Science, in- 
cluding mechanics, physics, chemistry, and all special phases of 
these exact and theoretical sciences; Shop; Engineering, includ- 
ing all kinds of engineering, but chiefly electrical and some 
mechanical; and all Academic subjects, which is, in the case of 
engineers, only history and economics. 

Comparison of Technical School Grades with Other Criteria.— 
The first question to raise about the grades is in regard to their 
relation to success in industry or practical work. These en- 
gineers had not yet been in full-time work, but their year of 
work in the Educational Department and in the shop under the 
direction of that department, could be considered as one form of 
practical work in which success is to be determined. ‘The esti- 
mates made by the committee of the Educational Department, 
explained under the title, “Criteria of Success,” might be con- 
sidered as one rating of success in the Educational Department. 
Grades given in the Educational Department classes, and grades 
given by the foremen on the shop work, can be considered as 
another check. Ratings on various qualities, explained in a sec- 
tion later, could be considered as the success of the men in im- 
pressing the foremen with their ability, and might be related to 
school grades. Also, the correlation between technical school 
grades and the ratings of the men made by the representatives of 
the Educational Department at the time of the employment inter- 
view, might be some measure of the extent to which the two 
measure the same qualities. Finally, a positive correlation be- 
tween mental alertness or general intelligence test and college 
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grades would be at least some evidence that school grades were 
some indication of what is measured by tests as general intel- 
ligence or mental ability. The correlations between technical 
school grades and the various other measures or criteria are as 
shown in Table II. 


TABLE II. 


Correlations of Technical School Grades with Other Measures. 
No. Correlation 
ofcases (r) 
Technical school grades with estimated intelligence, 


CUR EE nav cccnecasmneceeanbebenbeceue 88 0 
Technical school grades with educational department 2 

CUR INS Fa Dako 0 bbw ore cedbnda eee dibes sade 60 0 
Technical school grades with foremen’s shop grades... 62 0 
Technical school grades with foremen’s ratings on traits 85 0 
Technical school grades with ratings at interview for 

CUE 5b rk ba ko ksb8 5 0 <4 0 cde OR GUkeeews hos eks 21 +.22 
Technical school grades with Bureau Test VI.......... 56 +.37 
Grades in languages with foremen’s ratings on traits... 59 0 
Grades in mathematics with foremen’s ratings on traits.. 72 Ke) 
Grades in shop with foremen’s ratings on traits....... 31 0 
Grades in engineering with foremen’s ratings on traits.. 66 0 
Grades in academic subjects with foremen’s ratings on 

COINS Mike cai eien civ and on. sas ddan eee sahewias 64% 44 0 
Grades in languages with Bureau Test VI............. 43 O 
Grades in mathematics with Bureau Test VI........... 54 +.34 
Grades in shop with Bureau Test VI.................. 24 +.67 
Grades in engineering with Bureau Test VI.......... 53 +-.26 
Grades in academic subjects with Bureau Test VI...... 35 0 


The correlation coefficients between technical schocl grades 
and other criteria, particularly the Company’s Educational De- 
partment class grades, shop grades, and shop ratings, might be 
taken to indicate that the technical school grades are unreliable. 
However, I do not believe that this conclusion is justified. 
Rather, I believe that the shop and class grades, and the ratings 
are unreliable, or are largely the cause for the lack of correlation. 
The cause for this belief will be shown later in this report. 

The composite of technical school grades shows a significant 
correlation with intelligence test scores. Doubtless the correla- 
tions would be higher if the grades were not from so many dif- 
ferent schools with different standards. I believe that in any 
particular school, an average or composite of the grades received 
by each student would rank the students in general ability as well 
as any measure; but the standards of work and the methods of 
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grading are so incomparable in different schools that they can 
not be used by the employer to compare men from different 
schools. A standardized psychological test would show the same 
thing in much more reliable and usable form. That is, a general 
intelligence test given to a group of engineers selected from 
sixty different technical schools would give a more reliable rank- 
ing of the men according to their ability than a comparison of 
grades from these different schools; and the ranking could be 
obtained much easier by a test than by an attempt to transmute 
various systems of grades into a common scale. 

The intercorrelations of the grades in the various subjects is 
shown in Table III. As might be expected, the correlation be- 
tween closely related subjects, such as Mathematics and Engin- 
eering, (r == +.57) is greater than between what are usually 
considered less closely related subjects, such as Academic Sub- 
jects and Shop (r = +.30). There might be some a prion 
question about the correlations between Shop and Mathematics 
or between Shop and Engineering, but the correlations are shown 
to be comparatively high, being (r—=-+.61) (r==-++.42) respect- 
ively. Shop grades correlate higher with Bureau Test VI scores 
than any other technical school subjects do; but the number of cases 
is so small that the large probable error makes the correlation co- 
efficient highly unreliable. However, it will be seen later that the 
design engineers, who are picked men and are supposed to be the 
most capable men in pure mathematics and engineering, have the 
highest grades in Shop, and are differentiated more widely from 
the other engineers by the average of their grades in Shop. It 
seems that the grades that a man receives in his shop work in 
the technical schools is significant of his special engineering apti- 
tude, and even of his general mental capacity. 

For differentiating the engineers for the different lines of 
work which they are to do, I thought that grades in different 
subjects might be significant. The average grade in each main 
subject for each class of engineers was computed, and curves 
plotted to show the relation of one group of engineers to the 
others. (See Diagram 1.) The design engineers are above the 
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Taste III. 
Intercorrelations of Technical School Subjects. 
Subjects of cases (r) 
Number CorreJation 

Languages with mathematics.................cee0. 77 +.38 
| SE IE i i vie ode Kcsveaanweds oeaeees 35 +.37 
Languages with engineering ...........ccccceeccees 69 +.55 
Languages with academic subjects ................. 53 +.30 
DE I BI wi ie ni ikecccceetecce.iccns 43 +.61 
Mathematics with engineering .................000. 85 +.57 
Mathematics with academic subjects ............... 57 +.42 
Ce od ee vceicuoweenueces Couns 63 +.42 
Shoo Wire MenGemic subjects .....ccccccacccccses« 21 +.30 
Engineering with academic subjects ...............: 54 +.35 


other engineers in every subject, indicating that they were su- 
perior students in the technical schools. The curves of the gen- 
eral engineers and of the operating, service, and works manage- 
ment engineers run comparatively close together, the average of 
one group in a subject rising or falling with the average of the 
other group of engineers. The average of the sales engineers 
varies inversely with the averages of the general and the oper- 
ating, works, and service engineers. The significant feature of 
the averages, however, is that the design engineers are very 
much higher in all subjects, except the academic subjects, com- 
posed chiefly of economics, while in this subject the sales en- 
gineers have almost as high an average, although the general or 
all-around ability of the design engineers is evidently higher. 
The man who has been selected for other reasons as promising 
of becoming a good sales engineer, is the one who had special 
ability in the social sciences. 


VI. EVALUATION OF GRADES AND RATINGS GIVEN DuRING 
TRAINING IN THE EDUCATIONAL DEPARTMENT OF 
THE INDUSTRIAL FIRM 


As soon as the engineers, usually just graduated from the tech- 
nical school, enter the employ of the Company, they begin the 
year of training in the Educational Department. Practically 
all the engineers spend the first two months taking the same 
courses and doing the same work. In addition to class work, 
they work about a month in one department of the shop and 
then pass on to another kind of work so as to become familiar 
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with as many phases of the manufacturing process as possible. 
To make the absorption of the men into the shop possible and 
efficient, the men have to be divided into many groups and sent 
into many different sections, but the work for all is very similar 
at first. At the end of two months, however, there is begun the 
work of segregating the men into groups according to the line 
of work that they will do permanently. That is, the sales en- 
gineers and the design engineers, works management engineers, 
general engineers, etc., are picked out. This process of trans- 
ferring the men from one department to another and of deter- 
mining to what line of work each will be eventually assigned, con- 
tinues until the end of the sixth month. Beginning with the sixth 
month a Sales School and a Design School are started; and the 
engineers begin specialized training for their future work. 

From the time that the engineers enter the Educational De- 
partment, extensive records of their work are kept. Most of the 
men are first assigned to industrial motor winding and are graded 
on that work, both in the classes and in the actual work in 
the shop. In this work, the correlation between the shop grades 
and the class grades was r = +.19, for 76 cases. The grades 
for any one group of engineers plotted separately showed no 
different correlations. This low correlation shows the disagree- 
ment of the impressions received by different people, and the un- 
reliability of opinions concerning traits in people, particularly 
when the judgment must be made only after a short period of 
observation of the subject’s work. The curves in Diagram 2, 
show that the design engineers receive the highest grades in 
the shops. This agrees with the fact that they received the high- 
est grades in Shop in the technical school grades. 

In addition to the grades in the different subjects or kinds of 
work, there come from the foremen over each department, rating's 
on various traits of the engineers working temporarily under these 
foremen. These ratings are on ten different traits or characteris- 
tics; and they are made in terms of A, B, and C. The names of 
the ten traits are reliability, industry, initiative, tact, attitude, 
analytical ability, aptitude, enthusiasm, personality, decision. 
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16 BRUCE V. MOORE 


Since the men are transferred each month and rated by different 
foremen, several ratings in each trait are recorded. By @ombin- 
ing these ratings a more discriminating rating on each man of the 
group being studied was obtained. That is, the values A, B, and 
C were transmuted into the values 2, 1, and 0, respectively ; and by 
combining and averaging these ratings, fractional values be- 
tween these integers were obtained, such as 0.8 or 1.3 or 1.7 as 
a man’s average rating. Correlations of these shop ratings with 
other data on the men, are given in Table IV. 


TABLE IV. 
Correlations of Shop Ratings with Other Criteria. 
Number Correlation 


Subjects of cases (r) 
Shop ratings with estimated intelligence, committee’s 
CONE sin ica aby BOR Dd tron 9 44 os Aken cteeeakis 95 0 
Shop ratings on analytical ability with estimated in- 
telligence, committee’s rating..............ceeee8: 04 .O 
Shop ratings with Bureau Test VI scores.......... 58 0 
Shop ratings on analytical ability with Bureau Test 
VE IO saw cae reek 6 0865 Nai Saws ChaN 68 seb es 56 0 
Shop ratings with ratings at interview for employ- 
aR ere eee Pd Sen ane eS a err eee 27 .O 
Shop ratings on analytical ability with ratings on in- 
telligence at interview for employment............ 26 0 
Shop ratings on personality with ratings on personal 
qualities at employment interview................ 27 +.25 
Shop grades with shop ratings ...........ceceeeeees 74 0 


From these correlations we conclude that the ratings made 
by the foremen on the men working under them only one month, 
are very unreliable, and really indicate nothing. Doubtless, this 
unreliability cannot all be charged to the inability of the fore- 
men to judge the men on some of these traits; but the method 
of rating, of recording these ratings, and of combining ratings 
by different foremen, makes them incomparable and unreliable. 
In the shop and the Educational Department, the interpretations 
of analytical ability and of aptitude are practically the same as 
that which is called general intelligence by the psychologists. The 
ratings on analytical ability were correlated with estimated intel- 
ligence and with test scores in the hope that a significant positive 
correlation would be found, but there was none. One exception 
to the lack of positive correlation is the small correlation be- 
tween personality as judged by foremen and personal qualities 
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as judged by the representatives of the company who rated the 
men af the interview for employment. It seems reasonable to 
expect such a positive correlation; for it is doubtless the general 
personality which influences both the interviewer’s and the fore- 
man’s rating on a man in most of the traits, in much the same 
way. 


TABLE V. 


Some Intercorrelations of Ratings by Foremen on Traits of Engineers. 
Number Correlation 


of cases (r) 
Ret ee SE ao oo ncn cede bdebbacwouns 117 +.66 
Jet IE is pines cdscccgnadvabcbasiass ows 113 +.66 
TnstiAGeO SEM GUIMIRGMA go. oo ces covnccccvcescces 107 +.42 
Analytical ability with aptitude ...............0000. 102 +.72 
Analytical ability with personality ................. 104 +.26 
Enthusiasm with personality ..........cccsseceeees 106 +.57 


Correlations between the ratings on some of the traits were 
computed. The highest correlation found is between analytical 
ability and aptitude, + .72, which seems a priori reasonable. The 
next highest correlation is (r == ++-.66) between reliability and in- 
dustry, which might be expected; but there is the same correla- 
tion between tact and initiative, two traits which a priori we 
should judge would tend to be found in somewhat opposite 
types of individuals. The lowest correlation is between analytical 
ability and personality, which does not seem unreasonable. 

To discover whether the ratings on the ten different traits 
could be of any value in differentiating the engineers, the average 
of the ratings on each trait for each group was computed. These 
averages are shown in Diagram 2. The curves tend to follow 
each other very closely. Also, it was found in most of the rat- 
ings that if a man was rated high in one trait, he was rated com- 
paratively high in another. Personality and enthusiasm differ- 
entiate the groups most widely. Frequency column diagrams, 
Diagrams 3, 4, 5, and 6, show that the ratings in these two 
traits and in tact and initiative, tended to separate the sales en- 
gineers, and the work management engineers from the other 
engineers. By combining the ratings in these four traits we get 
a very definite differentiation of the four groups of engineers. 
See Diagram 2. It seems evident that people on coming in con- 
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sonality by Foremen of Student 
Engineers Classified by Occu- 
pation. 
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tact with the different groups of engineers sense a difference in 
the personal traits of these men; but definite analysis and measure- 
ment of this difference is difficult in any particular individual. 


VII. A Strupy or Ratinc SCALES 


For differentiating sales engineers from other engineers, it is 
probably true that the personal qualities aside from intelligence 
are more important. This has been recognized in the attempt 
to estimate these qualities as a whole without a rating scale when 
a man was being considered for any particular work. There are 
at present no psychological tests for measuring these qualities, 
but there are refined methods of making estimates or ratings. 
The rating scale is a tool or means for carrying out such a more 
accurate method. Its value as a measure of various traits de- 
pends upon the standards which it sets up as measuring rods, and 
its accuracy depends upon the definiteness of these standards. 


1. Principles of a Rating Scale 


Employers have always tried to estimate the value of a man as 
a whole, considering his qualities altogether without the use of a 
rating scale. Great differences in the estimates made by different 
persons on the same man have shown this to be an uncertain 
method. The man’s qualities must be separated and rated by 
themselves; for if they are not, the estimator is likely to let his 
prejudices consider only one prominent quality and let them bias 
his judgment so as to neglect all other qualities. For example, 
a foreman or supervisor may have an intense dislike for one of 
his workers because the worker has a disfigured face, and un- 
consciously the foreman overlooks the man’s other good quali- 
ties. However, when the rating scale calls for a consideration 
of these qualities separately, he is forced to realize that the 
worker may be intelligent, industrious, skillful, and codperative. 
After the foreman is led to face the facts in this way, he must 
admit that, after all, the man has good points that are valuable in 
a workman. On the other hand, the foreman may have to admit 
that he has been overlooking another workman’s laziness and 
lack of skill because this man was always good-natured and so- 
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ciable. Moreover, the fact that these ratings must be signed and 
given in black and white to a superior who can check them, 
leads the foreman to consider sources of inefficiency in his work- 
ers. It also makes him more careful and fair-minded in his judg- 
ments of his men. 

We can then lay down as the first principle of the rating scale 
that: (1) Instead of trying to estimate a person’s ability as a 
whole for doing any particular work, this ability is analyzed into 
component essential abilities or traits, and each trait is rated inde- 
pendently of the others. The traits selected should be the most 
important for success in the particular:situation. It is much 
better to have a few traits that are really essential and critical 
for a worker’s success than to have ratings on many traits, some 
of which are relatively unimportant. That is, it is much better 
to concentrate on a few important qualities and have ratings 
on these made and really used than to have a system so elaborate 
that it breaks down from its own size and complexity. The traits 
included in the rating scales discussed here were selected for 
rating men under certain conditions for a definite purpose. Any- 
one using this scale would have to consider these traits for their 
particular purpose in the light of the principles stated here. 

The reliability of a rating scale as a measure of various traits 
depends upon the standards, particularly the definiteness of the 
standards which it sets up as measuring rods. Therefore, other 
principles for making an accurate rating scale are to be observed; 
and the next principle is: (2) the different traits determined 
upon must be really different and as distinct from each other as 
possible. They must be supplementary to each other with the 
minimum of indefiniteness and overlapping. 

(3) For rating a person in any trait, the person doing the 
rating must be acquainted with the one to be rated, and have 
more or less dependable facts for making a decision. When an 
employee asks for a promotion or transfer, the employment 
manager cannot rate the employe in codperation or in industry; 
but the worker’s previous foreman probably could. On the other 
hand the employment manager might be a better judge of the 
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applicant’s appearance and manner in considering him for trans- 
fer to the office or some place where he would meet the public. 

(4) The traits must be as sharply defined as possible so that 
different people doing the rating will rate the same trait. In 
naming and defining traits, use terms which are more or less 
common and have, as nearly as possible, universally the same 
meanings accepted by everybody and the minimum chances of 
ambiguity and misunderstanding. 

(5) For rating an individual in a trait, the basis of comparison 
used as a scale should be as concrete and as familiar as possible. 
To provide this condition, the person to be rated is compared in 
regard to a particular trait with other well-known persons who 
differ in the extent to which they have that particular trait. That 
is, a foreman, in rating a man, should compare him with men 
who are doing the same kind of work under similar circum- 
stances; and the foreman must be well acquainted with the traits 
and qualifications of the men. 

(6) Where more than one individual is to be rated in more 
than one trait, more comparable results are obtained by rating 
all individuals in one trait before rating any of the individuals 
in any other trait. 

(7) More reliable results are obtained by having a person 
rated by more than one person, ratings by three persons being 
recommended if it is possible to get them. These ratings should 
be made independently and then averaged. A revision of one 
rating by another person is not so accurate as a combination of 
two ratings made separately. 


2. Evaluation of the Rating Scale Used, Entitled, 
“Rating by Interviewer’ 

With the above principles in mind, a study was made of the 
results of the scale already in use by the Company, which is en- 
titled Rating by Interviewer (p. 24). The purpose was to 
check the value of such a scale and the particular selection of 
traits. The scale was used with only thirty-one students, but the 
number is sufficient to give results that are significant, though by 
no means final. Table VI. shows the correlations between the 
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RATING BY INTERVIEWER 
THIS SIDE NOT TO BE WRITTEN UPON BY APPLICANT 
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I. Physical qualities—Physique, 
bearing, meatness, voice, energy, 
endurance. (Consider how he im- 
presses men in the above respects.) 

















II. Intelligence—Accuracy, ease 
in learning, ability to grasp the 
point quickly, to express himself 
clearly, and to estimate a new sit- 
uation. 




















ITI. Leadership — Initiative, 
force, decisiveness, tact, helpful- 
ness, ability to inspire men and to 
win their loyalty and cooperation. 




















IV. Personal qualities—Indus- 
try, dependability, loyalty, readiness 
to shoulder responsibility for his 
own acts, freedom from conceit and 
selfishness, readiness and ability to 
cooperate, and charm of personal- 
ity, breadth of conceptions. 


























V. General Value to the Com- 
pany—Special talent (commercial, 
mechanical, mathematical, execu- 
tive), gets results, works well in 
an organization. Special interest 
in W. E. & M. Co. Previous train- 
ing and experience. Will develop. 












































Highest 20 Remarks 
High 16 

Average 12 

Low gs 

Lowest 4 | 
Highest 20 

High 16 ee 
Average 12 

Low 8 : 
Lowest x | 
Highest 20 

High 16 

Average 12 

Low 8 

Lowest 4 | 
Highest 20 

High 16 

Average 12 

Low 8 

Lowest 4 | 
Highest 20 , 

High 16 

Average 12 

Low 8 

Lowest 4 | 





Recommendations of Professors and Instructors— 














REPORT OF INTERVIEWER 


1. General impressions 














Work best suited for 





> WwW LO 


Acceptance (write Yes or No) 
Next action promised by interviewer 











Signed 


Date 





Interviewer 








PERSONNEL SELECTION OF GRADUATE ENGINEERS 25 


original interview ratings, the tests, and the ratings made by 
the Educational Department committee one year later. Also 
the relation between each trait and the total of ratings on each 
trait, and the intercorrelations between traits is shown. All cor- 
relations are positive. 


TABLE VI. 


Correlations of Ratings by Interviewers with Other Criteria. 
Number Correlation 


of cases (r) 
Interviewers’ composite ratings with estimated in- 
telligence, committee’s rating .........eccceeeeees 31 +.65 
Interviewers’ composite ratings with Bureau Test 
{sO Pe CPSP EET Teer err er ore re rere 18 +.58 
Interviewers’ ratings of intelligence with Bureau Test 
VE SOR 6 Ga bons ok bs eckbwe ees beeen een eke 18 +.57 
Interviewers’ ratings of intelligence with estimated 
intelligence, committee’s rating...........eeeeeees 31 +.22 
Interviewers’ composite ratings with shop ratings.... 27 0 
Interviewers’ ratings of intelligence with shop ratings 
Oy SEE 6s. 90 a50ds0sessceneueuaneaases 26 0 
Interviewers’ ratings of personal qualities with shop 
RR ES 6. n6 0 00 0095dntwdaesaewioane 27 +.25 
Interviewers’ composite ratings with technical school 
CI rain kbh +4400 shoo ke end ceaetenanass 21 +.22 
Interviewers’ composite ratings with their ratings of 
PIED ARs Sila 5 Ki b9' <0 400040 9 ecenatanaenawlee 32 +.80 
Interviewers’ composite ratings with their ratings of 
NR I. . c,d w'ces 0.0 0 0 Snd dae ee aeene 32 +.79 
Interviewers’ composite ratings with their ratings of 
I on dois adne «owen akbeeiiiie kes 32 +.73 
Interviewers’ composite ratings with their ratings of 
SOMITE GHEE WO COUNITINT i. on ice cc iddccbn sewer ce 32 +.68 
Interviewers’ composite ratings with their ratings of 
CS eo ho ne ei dhaeccc sdb Reeehae ass 32 + .68 
Personal qualities with general value to company.... 32 +.67 
Personal qualities with leadership ................. 32 +-.59 
Intelligence with general value to the company ...... 32 +.48 
Leadership with general value to the company ...... 32 +.47 
Leadership with physical qualities ................. 32 +-.46 
Personal qualities with physical qualities ........... 32 +.45 
Intelligence with personal qualities ................. 32 +.44 
Physical qualities with general value to company.... 32 +.43 
Intelligence with leadership :..........sseceeeeeees 32 +.40 
Intelligence with physical qualities .............6+.. 32 +-.23 


The interviewers’ ratings of intelligence agree much better 
with the scores of the psychological test for intelligence, r = 
+.57, than they do with the committee’s later estimate of intel- 
ligence, r = +.25; but since the interviewers’ ratings of intelli- 
gence correlate comparatively low with their composite ratings 
including other qualities, r — -+.68, and since the composite 
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ratings have high correlation with the later estimate of intelli- 
gence by the committee, r = -++.65, we might take these facts 
as evidence that qualities other than intelligence have uncon- 
sciously been included in the later ranking of the men. 

The most important correlations to consider in relation to the 
rating scale, are those of Personal Qualities, which correlate, 
with General Value to the Company, r = +.67, and with Lead- 
ership, r == +.59. These correlations are entirely too high be- 
tween traits which according to the best principles should be 
disparate traits. These high correlations are undoubtedly due to 
the indefinite and too inclusive definitions of these traits. There- 
fore I have worked to make these traits more definite and ex- 
clusive in the revised definitions or descriptions of them. The 
revised forms are seen in the Interviewer’s Master Scale, dis- 
cussed later. Some of the other traits have had the definitions 
slightly changed to make the same improvement toward definite 
unit traits which could be more accurately rated. 


3. The Revised Form of Interviewer’s Rating Scale: 
The Interviewers Master Scale 


Not only were these results considered in the revision, but the 
whole aim was to make the rating scale best adapted to select 
men for the particular lines of work to be done for the particular 
Company. From the occupational description or information that 
had been obtained concerning each line of work to which a grad- 
uate engineer is later assigned, I made an analysis of what quali- 
fications or traits are essential for success in each line of work 
or specific occupation. The traits for each occupation were 
listed separately. Then to enrich this list and to be sure that no 
traits were omitted, comparison was made with the list of traits 
already being used by the Westinghouse interviewers and also 
with the traits considered in a rating made later in the Educa- 
tional Department. Also all other available rating scales, includ- 
ing the Bureau Scale for Salesmen, and the Army Officer Rating 
Scale, were compared with the list of traits. Finally, Mann’s 
Study of Engineering Education, and Davenport’s Trait Book, 
which lists several hundred traits, were examined for suggestions 
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of traits. From the final nine lists of traits (one list for each 
occupation), selection was made for the traits common to all 
lists and those which were similar were grouped under one name 
of an essential trait. Then selection was made of those traits 
which were not common and which tended to differentiate the 
occupations. For making the scales to be used in the first selec- 
tion and employment of applicants or prospective applicants, the 
common and the differentiating traits were combined. For con- 
structing the scales which were to be used later in separating the 
men, the differentiating traits were emphasized. 

The first work in preparing rating scales was to revise the scale 
already in use, called the /nterviewer's Rating, shown on page 24. 
As the revisions were simply in the definitions of the traits, these 
may be seen in the form for the Interviewer’s Master Rating 
Scale, which was prepared. This Master Scale is the standard 
or measuring rod with which the senior engineer is compared 
and rated by the interviewer. The form of this is shown on page 
28. The method of making and using the scale is explained in 
the following section. 


4. Instructions to Interviewers for Rating 
Senior Engineers 
Significance of the Rating Scale. 

(1) The rating of a senior engineer is a numerical expression 
of the degree in which he possesses five qualifications consid- 
ered essential in a Westinghouse engineer, namely: (I.) Physical 
Qualities, (II.) Intelligence, (III.) Leadership, (IV.) Social and 
Personal Qualities, (V.) General Value to the Company. The 
rating is made by comparing him in each of these respects with 
engineers who are now employed in the Company. Each rater 
makes his own scale, using the Interviewer’s Master Rating Scale. 
Proper rating is largely dependent on the possession of an ac- 
curate Master Rating Scale. Do not start to prepare your scale 
until you can give at least thirty minutes to it. 








we” 





28 BRUCE V. MOORE 


INTERVIEWER’S MASTER SCALE 


(To be filled out by all interviewers in committee together, for use as a 
standard in rating each applicant.) 


Have before you the names of at least twenty-five of your engineers and 
salesmen with whom you all are well acquainted. Include in this list 
men that rank highest, lowest and intermediate in Physical Qualities; men 
that rank highest, lowest and intermediate in Intelligence; and similarly in 
Leadership, in Social and Personal Qualities, and in General Value to the 
Company. 


I. PHysicat QUALITIES III. LeapDERSHIP 
Disregard every characteristic of Consider force, decisiveness, enthus- 
each of your engineers and sales- iasm, and ability to inspire men and 
men except their physical qualities. to win their loyalty and cooperation. 
Consider physique, bearing, neat- Highest Mr.......csescsccecces. 20 
ness, facial expression, voice, charm High + Mr.......sccesscecseess 16 
of appearance and manner, energy, PE SU nc enc cedu deaeee es oc 12 
and endurance. (Consider how he Low ER re ee 8 
impresses men in such qualities. ) NE Fs dies cackdeGSducwess 4 
PUNE Ws oe Gise cc be eck ces cnc 20 
SS SO eee 16 IV. Soctar AND PEeRsoNnaL QUALITIES 
aed c cn teenes 9% 12 Consider conversational ability, tact, 
Low ES Ses Re 8 freedom from conceit and selfish- 
CO SC cet kcnne es dsoane 4 ness, readiness to shoulder responsi- 
bility for own acts, readiness and 
Il. INTELLIGENCE alulity to cooperate, sociableness, 
Consider common sense, judgment, and congeniality. 
ability to grasp the point quickly, to Flighest Mir.......0s.0s0¢ eee 20 
express himself clearly, and to esti- High = Mr... esses eeeeeeeeeees 16 
mate a new situation. On the basis Middle Mr ee 12 
of intelligence make a selection of Low MF... ee eee e eee eens 8 
five of your men as in Number I Fe a TET ERR eT LTE 4 


above,—highest, lowest, middle, 
high, low. These five need not be. V- GENERAL VALUE TO THE COMPANY 


the same as those in Number I. Consider previous training and ex- 
PENG ashes ie keg sank 20 perience showing interest and adap- 
PE Sede debubhan soe vee 16 tability, (commercial, mechanical, 
DE a kiki boeksstenened 12 mathematical, executive) special in- 
Low ie peRiiekcevteees 8 terest in W. E. & M. Co. and pro- 
Ri I iio de eas be sccsces 4 fessional attitude. 
SE Mine cksbinnsoduceces +e 20 
High SD Sv cs ddan kmewees 16 
BE es 6 ois cd saebinbe sss 12 
Low SP Pre eee 8 
EOE Mik s 60} cbaoeen see 4 


How to Make the Master Scale. 


(2) Write on small slips of paper the names of from 12 to 25 
engineers who have passed through the Educational Department 
within the previous five years. They should be men with whom 
you are well acquainted, and of whose degree of success you have 
some definite knowledge. Include some whose qualifications are 
extremely poor as well as those who are highly efficient. If these 
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names do not include all the grades for each of the five qualifica- 
tions, others may be added. 

(3) Look over your names from the viewpoint of Physical 
Qualities only. Disregard every other characteristic of each 
man except the way in which he impresses men by his physique, 
bearing, neatness, voice, charm of appearance and manner, en- 
ergy and endurance. Arrange the names on the slips of paper in 
order from highest to lowest on the basis of the Physical Quali- 
ties of the men. Select that engineer who surpasses all the others 
in this qualification and enter his name on the line marked 
Highest under Physical Qualities on the Interviewer’s Master 
Scale. Then select the one who most conspicuously lacks these 
qualities and enter his name on the line marked Lowest. Select 
the man who seems about half way between the two previously 
selected and who represents about the general average in Phys- 
ical Qualities; and enter his name on the line marked Middle. 
Select the engineer who is halfway between middle and highest, 
and enter his name on the line marked High. Select the one who 
ranks halfway between middle and lowest, and enter his name on 
the line marked Low. 

(4) In the same manner make out scales for each of the other 
four qualifications (Intelligence, Leadership, Social and Personal 
Qualities, and General Value to the Company ). 

(5) Each engineer whose name appears on the Scale should 
be one who exhibits clearly and distinctly the qualification and 
the degree of the qualification for which he has been chosen. 

(6) The names for Highest and Lowest on each section of 
the Scale must represent extreme cases. The names for the 
Middle should be that of an average engineer, halfway between 
extremes. High and Low should be halfway between the Middle 
and the extremes. An even gradation of merit is important. 

(7) In making or using any section of the Scale, consider 
only the qualification it covers, totally disregarding all the others. 


How to Use the Scale. 


(8) Rate your senior student engineer for Physical Qualities 
first. Consider how he impresses men by his physique, bearing, 
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neatness, voice, charm of appearance and manner, energy, and 
endurance. Compare him with each of the five men in Section I. 
of your Master Rating Scale, and give him the number of points 
following the name of the man he most nearly equals. Check this 
number under Physical Qualities on your blank entitled Inter- 
viewer's Rating. If he falls between two men in the Scale, give 
him a number, accordingly, between the numbers of these two 
names in the Scale. 

(10) Rate the engineers in a corresponding manner for each 
of the other four essential qualifications. 

(11) In rating, make a man-to-man comparison of the man 
with the men whose names appear on your Scale—never in 
terms of numbers directly. Disregard the numerical equivalent 
until you have made these concrete comparisons. 

(12) When rating several engineers, rate all of them on each 
qualification before adding the total for any one. 

(13) This is not a percentage system and you should not 
allow yourself to fix in mind any particular number of points 
you think the engineer ought to get. 

(14) The total rating for an engineer is the sum of the ratings 
you give him in the five separate qualifications. 


5. College Instructor's Rating Scale. 


There are many traits desirable in an engineer which can not 
be rated with any reliability from a brief interview with a senior 
student. However, an instructor having had the student in his 
classes and knowing of his college activities, could rate the man 
on definite objective criteria. It was thought at first that the 
College Instructor's Rating Scale (see page 31) should have 
entirely different traits; but in order to reduce the possibilities 
of confusion to the minimum and to make the combination of the 
ratings of the two scales easy, practically the same names of 
traits are used on both scales. The ratings from the Instructor's 
Rating Scale can be checked just to the left of the interviewer's 
ratings on the present form, Rating by Interviewer. However, 
it is ¢mportant that the interviewer make his rating independently 
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COLLEGE INSTRUCTOR’S RATING SCALE 


Will you please rate the student named below for the traits indicated, 
keeping in mind employment after graduation. Give the rating independently 
without consulting others. This rating sheet is to be returned to the Bureau 
of Recommendations without delay. Do not rate any student whom you 
have not had in class at least one term. 

NG dw Ed b0b5.00.06 9008 4apnscthderdbeitannaas 

Among the members of the average senior class in this student’s CoURSE 
AND SCHOOL, in which fifth would this student rank in each trait? Indicate 
the rank in each trait by placing a check (V) at the proper height in each 
column, grading the student as finely as you can. The traits are described 
as follows: 

I. Appearance and Manner.—Consider physique, bearing, neatness, facial 
expression and voice. (Consider how he impresses men in such qualities.) 

II. Intelligence—Consider accuracy, promptness, ease of learning, ability 
to grasp the point quickly and to express himself clearly, and ability to esti- 
mate a new situation and to get desired results professionally. 

III. Leadership.—Consider initiative, self-reliance, enthusiasm, tact, and 
ability to inspire men and to win their loyalty and cooperation. 

IV. Personal Qualities—Consider honesty, reliability, spirit of service, 
sense of responsibility for his own acts and moral stamina. 

V. Professional Interest—Consider industry, energy, perseverance, and 
conscientious application to work. 





Appearance Personal | Profession- 
and Manner|Intelligence|Leadership | Qualities | al Interest 





Highest Fifth 
Second Fifth 
Middle Fifth | 














Fourth Fifth | 




















Lowest Fifth | b LT yt 





To aid further the Bureau of Recommendations, please indicate, if possible, 
THE SPECIFIC KIND OF wWoRK for which this student is best fitted. 


ceceeceoeoe ee eee eee ee ese eee ee eee eeeeeeeeeveereeeeevreereeeeereeeeeeeeeeeeeeeeeeeeeeee 


SpecIFIC WorK 
In this Specrric Work how would you rate 
this student among the members of the aver- Highest Fifth: 
age — as :. this student’s oe Second Fifth: 
AND ScHooL? Indicate your opinion by a : - ray, 
check (V) at the proper height in the col- Middle Fifth : 
Fourth Fifth: 


umn to the right. : ss 
Lowest Fifth: 

















(Use the other side of this sheet for remarks concerning the student.) 





~os Aw 


ore, 


z 


Re Samtg nee 


Ye Ae 


a 8 
ae 


ae t= 
pad 


7 pease 
a 6 


2) > whale 


ty aes 2 


hws 
. + 
i 
+e 
o 
i 
; 
ne ik 
SF 
th ig 
+. 
¢ a 
ig 
; a 
a he 
2 
a. 
% ; 


2 eae 9 iam siaer ans 
eee ey F te 
aS 


menemerpenemee on 
Sea ee 

adien © ind Hite, wwe N es wn! om 
: $3 SITS ke OS. 
~ pt 
= ae be = 

+e. 

pice 





_ = * 


Se = ase 











32 BRUCE V. MOORE 


and before he sees the instructor's ratings or copies them on his 
sheet. Then an average of the two can be made. 

The traits to be rated are not defined the same on the two 
scales. They should be rated from two different viewpoints, 
namely from what the instructor’s acquaintance and observation 
reveals to be the actual rating of the trait, and from the first im- 
pression which the individual gives other people regarding the 
trait, as shown by the interviewer's rating. With the background 
and viewpoint of the instructor differing from that of the inter- 
viewer, the ratings will supplement and balance or check each 
other. Professional interest or industry and professional ability 
can be judged better by the college instructor; but General Value 
to the Company can be judged better by the Company representa- 
tive. 

The instructor cannot be expected to prepare a scale, so he is 
asked to rate the student in terms of fifths of the average senior 
class. The interviewers, however, should prepare an Inter- 
viewer’s Master Rating Scale, and by mental comparison of 
students with the men named on this scale, that is, a man-to-man 
comparison, a rating can be given to that student for each trait, 
and then be entered on the form, Rating by Interviewer. 


6. Shop and Class Rating Scale 


The unreliability of the shop ratings made by the foremen, and 
of the class and shop grades, has already been discussed in 
Chapter VII, entitled “Evaluation of Grades and Ratings Given 
During Training in the Educational Department of the Indus- 
trial Firm.” It was stated there that the causes of this unre- 
liability were evidently due to the lack of adequate definitions of 
the traits, to the lack of common understanding by the foremen 
of the meaning of the names of the traits, to the lack of a stan- 
dard, scale, or measuring rod with which to compare the persons 
being rated, and to the arbitrary method of recording and re- 
porting the ratings. In order to obviate or at least minimize 
these causes of unreliability of the ratings, the Shop and Class 
Rating Scale was prepared. It is intended that each foreman and 
instructor shall rate each student engineer under them once each 
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SHOP AND CLASS RATING SCALE 


I iets nd 960 36 056 0006 0.0 4ens eds a SR Pe mre rar 

Rate the man named above in the traits indicated. The traits are defined 
and described on the other side of this sheet. 

Among the GrapDUATE ENGINEERS whom you have known in this particular 
work that you have charge of, in which fifth,—highest, second, middle, fourth, 
or lowest, should this man rank in each trait? Indicate the rank in each trait 
by placing a check (Vv) at the proper height in each column, grading the man 
as carefully as you can. 
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Intelligence| operation | Industry | Leadership 
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NoTE:—Send to the Educational Department. 


DEFINITIONS OF TRAITS OR QUALITIES TO BE TREATED 


I. INTELLIGENCE.—Consider aptitude, ability to learn, common-sense, judg- 
ment, and ability to grasp the point, to express himself clearly, and to deal 
with a new situation. 

II. Cooprration.—Consider tact, courtesy, fair-mindedness, sociability, and 
ability to get along with other men and to fit into the organization. 

III. Inpustry.—Consider energy, perseverance, conscientious application, 
reliability, interest, enthusiasm, and loyalty. 

IV. LeapersHip.—Consider initiative, aggressiveness, force, decisiveness, 
and ability to influence men and win their loyalty and cooperation. 


month on this Shop and Class Rating Scale. The ratings are to 
be sent to the Educational Department office, where they are to 
be combined with the other data on the student engineers. 

For differentiating the sales engineers from the other en- 
gineers certain traits considered essential for sales engineering 
ability can be selected and the total rating on these traits for each 
graduate engineer will show his probable aptitude for sales 
engineering. To take advantage of data previously obtained, 
ratings on certain traits which differentiate the graduate en- 
gineers, for example traits desired in a salesman, might be se- 
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lected from the other rating scales and combined with these. For 
example, ratings on leadership, social and personal qualities, and 
physical qualities might be taken from the /nterviewer’s Rating 
Scale and from the College Instructor's Rating Scale. 


VIII. Survey AND EVALUATION OF INTERESTS AS 
CRITERIA FOR VOCATIONAL PLACEMENT 


1. The Significance of Interests 


The technical school grades, the grades and ratings on the 
work of the men during training in the Educational Department, 
and the results of the tests discussed in the next section, indicate 
that the engineer cannot be differentiated for the different kinds 
of work by general intelligence or mental alertness. Men with 
equal mental ability are found in all the lines of work. Suc- 
cess of a certain person in a particular line of work is evidently 
due to general intelligence or mental ability directed in a par- 
ticular line by interests. However, it is probably true that the 
interests are based on a particular ability or group of abilities 
which makes activity and achievement in a particular line of 
work possible and interesting. Leaving the question of general 
intelligence and particular abilities to be discussed in the next 
section, we consider here the interests themselves without rais- 
ing the question of what causes them. 

After the man has a definite strong interest in any line of work 
or activity, this interest, motivation, or whatever it may be, should 
be taken advantage of. It is difficult to discover just what the 
interests are, and often the graduate student engineers cannot 
decide themselves what work they should prefer. In order to 
make possible the planning and direction of their training, the 
custom has been to ask them to indicate their choice of work at 
the end of two months’ work in the Westinghouse Company, 
during which time they have been doing various kinds of work 
in the shop under the supervision of the Educational Department. 
To help them make this choice, and to help the executives know 
more definitely what this choice is, the form, Record of Inter- 
ests, which is a kind of rating scale, has been prepared. 
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2. Method of Using the Record of Interests 


With the exception of Section II., entitled Choice of Other 
Occupations, the whole of the Record of Interests blank may 
appear to add nothing new to the ordinary questions often asked 
when attempting to place a man. Nevertheless, there is a 
definite attempt to get these facts tabulated in usable form. Each 
class of information or facts concerning the man is considered 
separately, and a definite unmixed judgment is made on that 
phase of the man’s interests and qualifications. Thus the Record 
of Interests is really a rating scale of interests. The judgments 





RECORD OF INTERESTS 


Prepared by B. V. Moore, Research Fellow, Expressly for the Westinghouse 
Electric & Manufacturing Company, East Pittsburgh, Pennsylvania. 


To the Graduate Student Engineer: 


Answer the questions, and fill in the blanks as accurately as you can. None 
of your statements here commit you to any particular assignment. This 
record enables us to help you find the most suitable position in the Westing- 
house organization. 


I. PREVIOUS VOCATIONAL INTERESTS 


1. In your college work which type of work or subjects interested you 
most? Rank the types of work below in order of preference, numbering 
the best liked (1), the next best (2), and so on to the least liked, which 
you number (9). 


( ) English ( ) Field Work ( ) Testing 
( ) Economics ( ) Shop Work ( ) Laboratory 
( ) History ( ) Drawing or Draft’g ( ) Mathematics 


2. Did you ever construct or build any mechanical or electrical toy, appara- 
tus, or machines of any kind before you entered college? 
) No . ) ve 


eeeoeneeeveveeveeeveeveeeeveeveeeereeveeeeveeeeeeeeeeeeeereeeeeveeeeeeeeeeeeneeeeeeeeeeee 


4. Did you make these things chiefly because you wanted the things them- 
selves or because you enjoyed making them? 
( ) Wanted the things. ( ) Enjoyed making them. 
5. In making the toy, apparatus, or machine, did you make it like another, 
or did you design it yourself? 
( ) Made it like another. ( ) Designed it myself. 
6. Did you ever work as a salesman or clerk in a store, sell in a house 
to house canvass, or work as a salesman in any way? 


( ) Yes ( ) No Be 
7. What do you consider to be the most responsible position that you have 
ever held? 


8. In all your practical experience, including remunerative work during va- 
cations, which work or particular job have you liked best? 
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Il. CHOICE OF OTHER OCCUPATIONS 


Disregarding all your training and education and all differences in com- 
pensation and social standing of the following occupations, consider only 
your interest and satisfaction in working at each of the following occu- 
pations represented. Check with a plus (+) the ten kinds of work which 
you would most prefer to do; and check with a minus (—) the ten which 
you would dislike or least like to do. 
) Architect 
) Automobile repairman 

Automobile salesman 

Bank cashier 

Carpenter 


( 

= 

ig 

Pa 

( ) Draftsman 
(9 

£3 

ee 

( 


Machinist 

Newspaper reporter 
Pattern-maker 

Private secretary 
Purchasing agent 

Real estate agent 

Research worker in physics 


o~ 


Editor of popular magazine 


LOO LO OO LOO ON Lf ff 
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Hotel keeper or owner Stockbroker 
U. S. Governm’t astronomer Toolmaker 
) Lawyer Watchmaker 


III. AVOCATIONAL INTERESTS 
. What do you like to do as an avocation, hobby, or sideline? 


eeeeovreeeeeeeeeeeeeeeereeeeeeeeevneeeeveeeeeeeeeeeeeeaeveeeeeeeeeeeeveeeeeeenee 


2. What form of recreation or entertainment do you enjoy most in the 
evening after a day of study or work? 


*eeeeveoereeeeeeeeeeeeeeeeeveeeveeeeeeeeeeveeeceaeeveeeeeeeeveeeveeeeeeeeeeeeeeeaneene 
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5. Have you participated in any of the following extra-academic activities : 
Student debating ( ) Yes ( ) No 
Dramatics ( ) Yes ( ) No 
Student paper or annual book ( ) Yes ( ) No 
School politics ( ) Yes ( ) No 
Ever captain of a team ' 2) eae (" )} we 


IV. SOCIAL INTERESTS : 
. Name the social clubs, fraternities, and organizations to which you 
belong. 


Laan 


. Have you ever held any important office in any of these organizations? 

( ) Yes ( ) No 
How many times have you made an after-dinner talk?................ 
While you were in college did you prefer to live with a roommate or 
did you prefer to room alone? 

( ) Roommate ( ) Room alone 
Do you usually have a good time at smokers, conventions, and other 


gatherings of men? 
( ) Yes ope 
6. Excluding engineering magazines, name three magazines which you 


enjoy reading. 


eeeeeveeveeoevevneeneeeeeeereeeeeeeeeeeveveeereeeeeeeeeeeeeeeeeeeeeeeeeeevsee eevee 


V. INTERESTS OF RELATIVES 


N 


5 th 


yu 


1. What is or was the occupation of your father?............ccseceeeeeces 
2. What were the occupations of your: 
Cede EE errr err rrr rr Tyr rr Tr rrr rr reer 
ako Wad un wbine 0 605.006 b40 6 eb bg eNO EETES S000 


3. What are occupations of your brothers? 
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VI. TECHNICAL INTERESTS IN WESTINGHOUSE ORGANIZA- 


TION AND ITS WORK 


1. Considering all aspects of each line of work as you have become ac- 
quainted with them here so far, decide which particular kind of work you 
would prefer to make your life occupation. Give this line of work a rank 
or score of (1). Give your second choice a rank of (2), and so on, 
giving your least desirable choice a rank of (4). 

( ) Sales ( ) General Engineering 

( ) Works Management ( ) Design Engineering 
. Considering the content or subject-matter of the work itself, which sub- 
ject or courses in the W. E. & M. Co. Educational Department have in- 
terested you most? Write them in order of preference, the best liked first. 


(1) 


(2) 
. Check with a plus (+) the following lines of work which you think you 
would choose as your work in the Westinghouse organization. Check 
with a minus (—) those ‘which you think you would not choose as your 
work in the Westinghouse organization. 






































( ) General Engineering ( ) Railway Sales 
( ) Transformer Engineering ( ) Marine Sales 
( ) Control Engineering ( ) Stocker Sales 
( ) Switchboard Engineering ( ) Foreign Sales 
( ) Industrial and Railway ( ) Production 
Motor Engineering ( ) Time and Motion Study 
( ) Railway Equipment ( ) Inspection 
Engineering ( ) Cost Accounting 
( ) Material and Process ( ) Maintenance 
Engineering ( ) Operating 
( ) Research Engineering ( ) Dynamo Test 
( ) Detail Engineering ( ) Transformer Test 
( ) Power Engineering ( ) Large Industrial Motor Test 
( ) Industrial Sales ( ) Small Industrial Motor Test 
( ) Supply Sales ( ) Detail Test 
( ) Power Sales ( ) Farm Lighting Equipment 
Test 
(Do not write in the space below) 
I. II. | III IV. V. 
Vocational |Other Occ’s|Avocations | Social | Relatives Technical 
Design 
General 
Oper’g | 
Service | 
| 
Works 
Sales | | 














REMARKS: 
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or ratings are made by a man or by men who are intimately 
familiar with the lines of work and opportunities open to the 
student engineers. With these lines of work in mind, the one 
doing the rating considers the man’s answers to the -questions 
of Section I., entitled Previous Vocational Interests, and decides 
whether the facts appearing there indicate that the man is quali- 
fied by his fundamental interests and experience to be a design 
engineer, a general engineer, a works management engineer, an 
operating engineer, a service engineer, or a sales engineer. Then 
turning to the Swmmary on the last page of the Record of In- 
terests, the rater places a check ( \/) in the proper rectangular 
blank space under the column head, I. Vocational. Each section 
is considered separately in this way, a judgment is made from 
each class of facts, and a check is made in the proper square of 
each column of the Swmmary. By connecting the check marks 
with straight lines, a curve is obtained which shows the general 
tendency of the interests of the man, whether fitting him for 
design, general, works management, operating, service, or sales 
engineering. This gives a definite and permanent graphic record 
of pertinent facts concerning each man. Instead of a hazy vacil- 
ating general impression of the man, there is a tangible analyzed 
judgment of him. By keeping this on file and easily available, it 
is possible to have always before the executive considering the 
man, a careful record of previous judgments. 


3. Evaluation of Specific Questions in 
the Record of Interests 

To test out the value of the Record of Interests, it was filled 
out by sixty-four adults in one class and sixty in a second class 
of the School of Insurance Salesmanship at the Carnegie Insti- 
tute of Technology. The results for each Insurance Salesman- 
ship class are tabulated separately in order that the two sets of 
data will be a check on each other, and their agreement be some 
measure of their reliability. Later the Record of Interests was 
filled out also by thirty Westinghouse design engineers and 
thirty sales engineers, none of whom had been with the Com- 
pany less than a year nor more than five years. The following 
table shows the results obtained for some of the chief questions. 
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Each section is discussed later and evaluated separately in the 


light of the results obtained. 


Taste VII. 


Results Obtained from Record of Interests of Insurance Salesmen, Design 
Engineers, and Sales Engineers 


Insurance Salesmen 


Answered (Yes) Answered (No) No.of Cases 
First Second 
Class Class 


No. of First Second First 
Question Question Class Class Class 
I. 2. Constructed toy?..... 25% 40% 75% 
I. 4. Wanted toy? ........ 2770 53% 7390 
I. 5. Copied Design? ..... 69% 477% 31% 
I. 6. Had sold articles?.... 84% 81% 16% 
III. 5. Student Debating? ... 51% 56% 497 
SS eee 45% 48% 55% 
Student paper or book? 40% 52% 60% 


I. 1. Rank of Choice of School Studies or Subjects. 


Average Rank Given 


Second 
Class 
60% 
4770 
53% 
19% 
44% 
52% 
48% 


55 
15 


13 
62 
51 


53 
53 


62 
15 
15 
52 


Number of Cases 


First Second First Second 
Class Class Class Class 
ee | peepee 3.07 3.23 56 39 
RG as ic 40. coon os eck séa ee 3.54 2.92 46 38 
PEM Gs bck sk cctibiedds ews 3.60 3.29 53 45 
Fie SE in cunes da knscsuavck 4.13 4.39 24 28 
GO pe he coe cenceveuee 5.65 6.44 26 25 
Drawing and Drafting ......... 5.19 7.13 32 24 
TORN Cais 6 Side ccs encece 5.35 6.00 20 25 
L SE Wks 6 bc 06s bc cceebensns 4.97 5.13 32 30 
PERE iti iaikcnensseeasnee 3.47 3.60 53 39 
Design Engineers and Sales Engineers 
Number 
No. of Answered (Yes) Answered (No) of Cases 
Question Question Design Sales Design Sales Design Sales 
I. 2. Constructed toy? ...... 83% 50% 17% 50% 30 «8©=6.- 46 
re ~£¢ ff 3 eee 0% 37% 100% 63% 19 19 
I. 5. Copied design? ........ 12% 38% 88% 62% 17 16 
I. 6. Had sold articles? ..... 50% 82% 50% 18% 25 45 
III. 5. Student debating? ...... 32% 47% 68% 53% 25 45 
OS ee 16% 27% 84% 73% 25 45 
Student paper or book?. 36% 33% 64% 67% = 
I. 1. Rank of Choice of School Studies or Subjects. 
Average Rank Given Number of Cases 
Design Sales Design Sales 
sa chs oeaed ee ons 6.73 5.33 26 30 
ee e, dececaacte 6.04 4.63 26 38 
BUR aoe koa Vis vie viviees 6.32 5.55 25 36 
a, en aeeneias 5.54 5.13 26 38 
|S RE a a an 5.65 5.98 26 40 
Drawing and Drafting .......... 5.31 5.95 26 39 
Ws din 6 os o ks bp 0 da000% 3.73 3.05 26 40 
Re os cos Cec en cad 3.15 3.88 26 40 
PE ic cab evckewects 1.81 3.85 27 40 
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I. Previous Vocational Interests —Question 1, regarding col- 
lege work, proved to be of significance in differentiating the men. 
Both the insurance salesmen and the sales engineers, particularly 
the insurance salesmen, liked English, Economics, History, and 
Field Work better than the design engineers did. The design 
engineers liked Mathematics, Laboratory, Testing, Drafting, and 
Shop better than the sales engineers or the insurance salesmen 
did. 

Questions 2 and 3 showed some tendency for this differentia- 
tion, but they were not so valuable. Of the insurance salesmen, 
25% of the first class and 40% of the second class had con- 
structed toys; 50% of the sales engineers had constructed toys, 
while 83% of the design engineers had constructed toys. Ques- 
tion 4 showed a tendency of the salesmen to make things because 
they wanted them, while the design engineers made things be- 
cause they enjoyed making them. Those who made things be- 
cause they enjoyed making them, were 73% of the first class 
of insurance salesmen and 47% of the second class of insurance 
salesmen who made any things. 30% of the sales engineers who 
made things, and 100% of the design engineers who made things 
made them for the pleasure of construction. The other questions 
of this section were not so reliable, but they served to aid in form- 
ing a judgment or classification of the men. 

IT. Choice of Other Occupations.—In this list of occupations, 
there were fifteen occupations which were judged by fourteen 
people to be occupations that would be chosen or preferred by a 
man with engineering interests as primary. The other fifteen 
occupations were judged by the same people to be occupations 
that would be chosen by a salesman type of man. The occupa- 
tions were carefully chosen from a much larger list; and the 
judgments or classifications of the occupations by each of the 
fourteen people were tabulated so that any occupation concerning 
which there was not common agreement was detected and elim- 
inated from the list. It might be said that the occupations in- 
tended to be chosen by a sales type of person are comparatively 
higher-level occupations than those intended to be chosen by the 
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engineer type. This condition was allowed to exist, because all 
the men choosing these occupations are engineers and are already 
inclined to choose engineering type of occupations. By having 
them choose from a list containing very desirable sales type of 
occupations, there is a stronger tendency for the group to be 
differentiated. The Record of Interests as filled out by the sales- 
men and engineers contained this list of thirty occupations which 
were to be checked plus (+) or minus (—) according to the 
directions shown with them. The number of plus and minus 
checks made by each group for each occupation are shown in 
Table VIII. The same facts are shown in graphic form in Dia- 
grams 7, 8, 9. 


Tasie VIII 


Choices of Occupations by Insurance Salesmen, Sales Engineers and 
Design Engineers 


Insurance Salesmen 
Both Design Sales 


Occupation 1st Class 2nd Class Classes Engineers Engineers 
3 > = + = + — = 
AMR icticaoesean as ee 24 25 24 21 48 46 5 16 6 31 
ATG ae kbs re seeves 28 19 24 18 52 37 ie 3 29 10 
Auto repairman ......... IQ 25 8 27 a $3 6 5 I5 15 
Auto salesman .......... 52 8 46 3 98 II 10 6 44 I 
Pee eee 28 21 35 8 63 29 8 13 19 20 
Re oe eee 14 23 4 31 18 54 ee 9 22 
Captain of a ship ........ 23 2 22 16 44 39 14 8 21 13 
CN cS acdvbeewens 17 29 16 26 33 «55 . 25 15 
TNO ec iéic ce cves <a 20 24 43 55 6 17 7 33 
DR on ans sce 00 16 36 8 27 22 63 4.2 9 25 
Editor of magazine ...... o 6: 2 63 26 ae 19 17 
Hardware sales ......... 14 36 10 34 24 70 2 16 4 22 
i ee Ig 25 30 15 49 40 a a 19 618 
U. S. Astronomer ....... 9 28 I2 26 21 54 8 II 9 24 
[Fg Se ee ee 37 +10 390 4 76 14 9 9O 29 13 
RA i ks deste I 44 0 37 1 8&1 3 9 ae, 
Locomotive engineer ..... 20 19 i 27 31 46 - a 6 66 
RRS SESE 14 26 Ir 26 25 52 20 3 17 20 
Newspaper reporter ..... 38 «8 3 «66 75 14 5 14 25 9 
Pattern-maker .......... 5 39 : = 7 64 ~~ 6 7 23 
eee ka I 50 0 48 1 08 oo 2 0 43 
Private secretary ........ 38 «=O a3. 17 60 26 3 20 14 13 m 
Purchasing agent ........ 46 8 ee 8&7 II 9 8 4 2 af 
Real estate agent ........ a 6s es 89 II 6 16 2 7 if 
Research worker in physics 12 27 14 2! 26 48 a 22 13 od 
Se a 14 3! 9 21 23 52 5 15 $ 2 i 
ec cnws 53. 92 sm oy 31 49 7 18 12 25 mh : 
Stockbroker ............. 35 8 ae 70 #15 ag a 2 
C00 ae + 4 30 7 O 3 «8 3 25 h 
Watchmaker ............ 2 30 0 35 2 74 5 It 0 33 
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Diagram 7. { 
Rejection of (—) Occupations by Insurance Salesmen. 


men represented, 124. 
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Diagram 8. Percentages of Choices Showing Preference for 


(+), or Rejection of (—) Occupations by Design Engineers. 
Number of men represented, 28. 
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Diagram 9. Percentages of Choices Showing Preference for 


(+), or Rejection of (—) Occupations by Sales Engineers. 
Number of men represented, 50. 
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From the list of thirty occupations, twenty occupations were 
later chosen, because they had given the best results for differ- 
entiating ; and a new form was prepared as reproduced on page 
36. The student engineers are directed to check ten of 
them with a plus (+) as the more desirable, and to check ten 
with a minus (—) as the less desirable. The twenty occupa- 
tions classified as those chosen by an engineering type of man 
and those chosen by a sales type of man are as follows: 


Occupations Chosen by Occupations Chosen by 
Engineering Type Salesman Type 

Architect Automobile salesman 
Automobile repairman Bank cashier 
Carpenter Editor of popular magazine 
Draftsman Hotel keeper or owner 
U. S. Government astronomer Lawyer 
Machinist Newspaper reporter 
Pattern-maker Private secretary 
Research worker in physics Purchasing agent 
Toolmaker Real estate agent 
Watchmaker Stockbroker 


The relative percentage of the sales or engineering type of 
occupation chosen by a man indicates whether he is a sales type 
or an engineering type of man. The scoring method explained 
more fully is as follows: A stencil in the form of a cardboard 
with perforations or slots allowing only the engineering type of 
occupations to be visible is placed over the list of occupations 
which have been checked. The number of plus marks is counted 
and recorded in the margin. The number of minus marks is 
also recorded. ‘Then this stencil is removed and another stencil 
is placed over the list, allowing only the sales type of occupation 
to be visible; and then the number of plus and minus marks is 
recorded. The number of plus marks before engineering occu- 
pations is added to the number of minus marks before sales occu- 
pations in order to get the number of checks in favor of en- 
gineering occupations. The number of minus marks before en- 
gineering occupations is added to the number of plus marks before 
sales type of occupations to get the number of checks in favor of 
sales occupations. Finally the number of marks in favor of 
sales occupations is divided by the total number of check marks 
to get the percentage of marks in favor of sales occupations. 
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It would not be necessary to count both the plus and minus marks 
if all the subjects checked all the occupations; but this method of 
scoring can be used even though not all the occupations are 
checked. 

The Record of Interests papers of the salesmen and engineers 
were scored according to the above method; and the results with 
only the twenty selected occupations being considered are pre- 
sented here. Diagram 10 shows the percentages of sales occupa- 


ae 7 











































































































DES LIC UNEIER 
















































































4) — 

















%G 0-9 10-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90-100 % 


Diagram 10. Percentages of Choices Showing Preference for Occupations 
requiring Sales Type of Person. 

Considering the diagram as a four-fold table, the interest test (Record of 
Interests) places correctly eighty-two per cent of the engineers. ry = .84. 
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tions chosen by each group. The percentages of sales occupa- 
tions chosen are shown along the base line or x-axis. The height 
of the columns represents the number of men choosing the par- 
ticular percentages of sales occupations. For the insurance sales- 
men the percentages of sales occupations chosen were 73 per 
cent for the first class and 80.6 per cent for the second class. For 
the sales engineers the preference for sales occupations was 67.7 
per cent. For the design engineers the preference for sales occu- 
pations was only 30.0 per cent. 

This section, “Choice of Other Occupations,” is one of the 
most reliable sections of the Record of Interests; and no subject 
ever failed to fill it out. The men take an interest in choosing the 
occupations and expressing their preference. The engineers and 
the salesmen showed a definite tendency to like or be interested 
in occupations which in nature of work were similar to those 
they were already following. The kind of occupations which they 
thought would give them the greatest satisfaction to follow was 
a significant criterion of the kind of work in which they could 
be and were already successful. 

By computing for each engineer the percentage which his 
choices of occupations of a sales nature bore to his total number 
of choices, a definite measure of his sales engineering interests 
as opposed to design engineering interest was obtained. I as- 
sumed that if more than fifty per cent of a man’s choices were 
occupations requiring a sales type of person for success in them, 
that man was a sales type of person; and if more than fifty per 
sent of the man’s choices were occupations requiring a design 
engineering type of person, that man was a design engineering 
type of person. By this measure, 78 per cent of the sales engin- 
eers were of a sales type; and 82 per cent of the design en- 
gineers were of an engineering type. Or assuming that we did 
not know the actual occupations of the engineers, those en- 
gineers which this test of interests alone would select for sales 
engineering, would be 89 per cent correctly placed or classi- 
fied; and the men which this test of interest selected for design 
engineering would be 68 per cent correctly placed or classified. 

III. Avocational Interests.—Questions 1, 2, 3, and 4 did not 
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show any special differentiation except as they showed that the 
sales type of man was more interested in the social sports while 
the engineering type often enjoyed photography or some pro- 
fessional or technical hobby. Question 5 showed that there was 
a definite tendency for the sales type of man to take more active 
part in extra academic activities, (See Table VII). 51 per cent 
of the insurance salesmen of the first class, and 56 per cent of the 
second class, 47 per cent of the sales engineers, and 32 per cent 
of the design engineers had taken part in student debating. 45 
per cent of the first class and 48 per cent of the second class of 
insurance salesmen, 27 per cent of the sales engineers, and 16 
per cent of the design engineers had taken part in dramatics. 
40 per cent of the first class and 52 per cent of the second class 
of insurance salesmen, 33 per cent of the sales engineers and 
36 per cent of the design engineers had taken part on a student 
paper or annual book. 

IV. Social Interests —The answer to the questions on social 
interests were not significant in their nature in differentiating 
the men, except as they were a definite means of showing 
whether or not the man has social interests and is congenial. In 
answer to question 5, 60 per cent of the first class and 67 per 
cent of the second class of insurance salesmen, 77 per cent of 
the sales engineers, and 77 per cent of the design engineers pre- 
ferred a roommate. The other results of this section as a whole 
were consistent with the facts of differentiation shown in the 
other sections. 

V. Interests of Relatives —The interests of the relatives as 
shown by their occupations indicated a nature definitely similar 
to the choice made by the man considered, but there were excep- 
tions, so that the choice of occupations made by relatives could 
not be used as a criterion of the proper choice for the man, ex- 
cept as it might be consistent with and strengthen the judgment 
of a man’s interests as based upon the other facts of the total 
Record of Interests. 

VI. Technical Interests in the Westinghouse Organization 
and Its Opportunities—Here the man is asked for a definite 
statement of his particular interests in the Westinghouse Com- 
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pany. The results obtained in this section agree almost perfectly 
with the choice of kind of occupation indicated in the other sec- 
tions. 


IX. APPLICATION OF PSYCHOLOGICAL TESTS 
TO THE PROBLEM 


1. The Nature of Mental Ability and Its Problems 
for Tests 


By psychological tests, we mean those tests which are con- 
sciously based upon psychological principles and are for measur- 
ing a certain mental process or processes. Often such a test is 
intended to measure general intelligence. By general intelligence 
is meant not general information, but what is synonymously 
called menal alertness, mental capacity, or innate mental ability. 
In general, psychologists accept the theory that general intelli- 
gence is composed of a group of mental abilities or factors. 
Practically the same conception is held by the laymen; for when 
a layman judges or makes an estimate of another person’s gen- 
eral intelligence, he considers that person’s particular capacities 
and abilities all together, and approximates a sort of average of 
them which is compared with the averages of other people’s 
capacities and abilities. However, it is recognized that, except in 
the sense of a very broad general intelligence, people are indi- 
vidually different in that the general intelligence of one is com- 
posed of particular capacities and abilities different in degrees 
of perfection from those in the group of particular abilities com- 
posing another person’s general intelligence. That is, it may be 
said that there are different kinds of intelligences. These different 
kinds of intelligences are largely due to particular interests direct- 
ing the whole general intelligence in certain directions; but on 
the other hand, the interests are given substantial basis by par- 
ticular abilities which make activity in a particular line easy and 
interesting. In this way, what is practically considered as a 
kind of intelligence directed by particular interests, is a group 
of special abilities developed and integrated so as to be of prac- 
tical use in the life of the individual. If general intelligence or 
any kind of intelligence is analyzed very minutely, it is found 
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to consist of very particular abilities, which are neural bonds be- 
tween a particular situation and a particular response. Certain 
ones of these are essential for certain special abilities. If these 
essential very particular abilities are not present, the individual 
is not considered as specially qualified for the particular work. 
Other abilities or all of them together as general intelligence, 
may function vicariously in accomplishing to a lesser degree 
what is done by a special ability; but ability in that particular 
kind of work is limited by the weakness or lack of the ability 
peculiarly essential for marked success in it. It is the differences 
between individuals in the degree to which they possess these 
special abilities that differentiates them for particular lines of 
work. Such a special ability integrated and functioning with 
other abilities is a kind of intelligence, which, if tested, would 
yield a measure of potentiality for success in a particular kind of 
occupations or kind of work. Such a test may be a special ability 
test or it may be a trade test. 

Success in some kinds of work may require not a special ability 
but a certain amount of general intelligence, which is con- 
sidered as a certain potentiality in all of the large group of abili- 
ties usually functioning in the many activities carried on to a 
more or less extent by all persons. For an occupation in which 
greater general intelligence means so much greater capacity for 
success, persons could be differentiated by a general intelligence 
test. Such a test measures the abilities composing and therefore 
correlating with the average of the group of abilities called gen- 
eral intelligence. | 

These hypotheses are explained here as being the most per- 
tinent to the problem being studied. They are stated in the form 
of hypotheses, not as a solution to the problem, but as a restate- 
ment of the psychological questions raised before in the Introduc- 
tion. The method for attempting to answer the psychological 
questions, was to work in accordance with these hypotheses. 
First, a general intelligence test was given to ninety-four grad- 
uate student engineers. Later, a test was devised which was in- 
tended to test special ability for particular kinds of work. 
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2. The Results of a General Intelligence Test 


The general intelligence test used was the Personnel Bureau 
Test VI., which is a modified form of the Army Alpha general 
intelligence test. This test was given to ninety-four graduate 
engineers who were students in the Educational Department of 
the Westinghouse Electric and Manufacturing Company. Most 
of these men had been in the Educational Department from six 
months to a year. The men were already classified according 
to the kind of work to which they were to be assigned. The 
average of the test scores for each group is shown in Table IX. 


TABLE IX. 


Averages of General Intelligence Test Scores for Each Group of Graduate 
Student Engineers. 


Estimated Rank in 
Mean Median Intelligence, Com- 
Group Score Score mittee’s rating 
Dig OR doc ass scncctacns 154 150 12.95 
Operating, Service, Works ......... 140 144 84.2 
errr er Pe reer rs 137 135 59.9 
SaaS ET ins nchtadndcosnecda - 129 130 56.5 


The test scores for each group of engineers are shown graph- 
ically in Diagram 11, in which the height of each column indi- 
cates the number of men making any particular score shown on 
the horizontal axis. Although it has not been the intention to 
select the men with greater mental ability for any particular 
line of work, the results of the test indicate that there has been 
a tendency to select the men with greater mental ability for 
engineering, particularly design engineering. This agrees with 
the results of the study of the technical school grades. How- 
ever, the difference is not sufficient to differentiate the men, ex- 
cept that a critical score might be established, below which a 
man could not be accepted for design engineering. The cor- 
relations between this test and other tests and ratings are shown 
in Table X. 

On examining the cases in which the test scores do not agree 
with estimated intelligence, we find that there seem to be certain 
tendencies causing the disagreement. A table of the cases in 
which there is the greatest disagreement, and those in which there 





wy) 


BRUCE V. MOORE 


cy 
iS) 


Scores in Bureau Test VI 


70 80 90 100 110 120 130 140 150 
-79 -89 -99 -109 = -119 -129 -139 # -149 -159 


160 
-169 





Coe 
































Design 
Engineers 
















































































General 





Engineers 





Operating, 














Service, 


and Works 





Engineers 




























































































Sales 
Engineers 


Diagram 11. Scores in Bureau Test VI of Graduate Engineers Classified 


by Occupations. 
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TABLE X. 
Correlations of Bureau Test VI Scores with Other Criteria. 

Number 

Criteria Correlated with Test VI. of Cases 
Estimated intelligence, committee’s rating........... 74 
Composite of technical school grades ................ 56 
Technical school grades in languages ................ 43 
Technical school grades in mathematics ............. 34 
Technical school grades in shop ..............eeeee 24 
Technical school grades in engineering .............. 53 
Technical school grades in academic subjects ........ 35 
COMM III 25 Go bceS oecescvsecsccisnve se 58 
Ratings at employment interview .................6-. 18 
Rating in intelligence at employment interview ....... 18 
Bureau Test 10, Part I (Insurance salesmen Class 1).. 64 
Bureau Test 10, Part I (Insurance salesmen Class 2).. 58 
Bureau Test 10, Part I (Engineers) ................- 18 
Bureau Test 10, Part II (Engineers) ...............- 18 
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is the greatest agreement is presented for study and comparison 
of those cases. See Table XI. 


TasBLe XI. 


Cases in Which Test Scores Do Not Correlate with Estimated Intelligence 
and Cases in Which the Correlation Is High. 


Disagreement: High Rank by Test; Low Rank by Committee 


Number Estimated 
of Case Rank by Test Intelligence Rank Assignment 
I yy, 61 Engineering 
2 12 57 Sales 
3 18 63 Engineering 
4 21 62 Operating 
5 12 49 Engineering 
6 10 39 Sales 
Disagreement: Low Rank by Test; High Rank by Committee 
7 56% 13 Sales 
8 52 16 Sales 
9 52 17 Sales 
10 61 26 Engineering 
Agreement: Both below Average 
II 65 66 Sales 
12 63 64 Sales 
13 48 48 Sales 
Agreement: Both above Average 
14 I 4 Engineering, 
Design 
15 2 5 Engineering, 
Design 
16 RY, 6 Engineering 
17 84 9 Engineering 
18 12 12 Engineering 
19 24 25 Engineering 


In the group of those who received a high rank by the test, 
but a low rank by the committee, it is seen that most of the men 
in this group are engineers as contrasted with salesmen. These 
men tend to be more of the profound intellectual type as con- 
trasted with the bright attractive type. For this reason their 
pure reasoning or intellectual ability tends to be underestimated. 
It is also interesting to note that in the case of greatest disagree- 
ment, that of No. 1, the engineer was rated on the rating scale 
at time of employment, and he received in physical qualities only 
12 points out of a possible 20, in personal qualities only 12 points 
out of a possible 20, but in intelligence he received 20 points, the 
highest possible rating. 





why», 








54 BRUCE V. MOORE 


Of the graduate engineers who received a low rank by the 
test but a high rank by the committee, three are assigned as 
salesmen, and but one as an engineer. It is seen that there is a 
tendency for the man with high intelligence but comparative lack 
of highly-rated appearance and personal qualities to be under- 
estimated; but the man with good or pleasing appearance and 
personal qualities is over-estimated in intelligence. This is the 
usual tendency often found, because we tend to consider other 
qualities which are evident, but which, although valuable traits, 
are not intelligence as measured by the test. Of those cases in 
which there was perfect or most nearly perfect agreement, all 
those ranking above average were engineers, and all those ranking 
below average were assigned to sales work. It seemed very 
probable that in selecting men for engineering, particularly de- 
sign engineering, intelligence had been the chief qualification con- 
sidered; but in choosing men for sales engineering, other qualifi- 
cations had been given much weight. 


3. Construction and Evaluation of a Special Test for 
Differentiating Graduate Student Engineers 


(a) Hypothesis of the Test:—As was stated in Section II, the 
problem of this research was to discover or devise a method and 
means for selecting from the whole group of graduate student 
engineers those who would give their services most efficiently as 
sales engineers. The general intelligence test did not adequately 
separate the sales engineers from the other engineers. Evidently 
the difference between sales engineers and other engineers is a 
qualitative one more than it is a quantitative one in intelligence. 
Moreover, the difference is a relative one; for the purpose is not 
so much to pick a certain standard of sales engineers as it is to 
pick a certain number, about half of the group, who will be the 
best selection from that group for developing into sales en- 
gineers. For measuring this qualitative difference a general 
intelligence test could not be used. Moreover, any general intel- 
ligence test available was too easy for this group of men, because 
they all made high scores and were not sufficiently differentiated 
by it. Finally, the content or subject-matter of existing intelli- 
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gence tests was not relevant to the engineering profession, and 
therefore did not interest or appeal to engineers as a test might 
be made to do. For these reasons, another test was devised to 
distinguish two different kinds of intelligence. 

The theory upon which the test was constructed is that, of 
this group, the sales engineers on the one hand, and the design 
engineers on the other hand, are relatively at opposite extremes 
in the kind of intelligence possessed. The sales engineer as a re- 
sult of his innate particular abilities, his environment, and his 
training, has interests and abilities that make for an intelligence 
better fitted for the work of a sales engineer. He has developed 
habits of thinking and attitudes toward other people which are 
more like those of a salesman; and his interests and apperceptions 
have led him to pick up and retain more of the information, par- 
ticularly of a business, social, and human interest nature, which 
enables him to fit into the work of a sales engineer better than the 
design engineer type would. 

The design engineer type has innate abilities and interests 
which with environmental influences, experiences, and training, 
have developed into an intelligence better fitted for the work of 
a design engineer. He worked intensively at his technical prob- 
lems in college, and has developed habits of thinking and atti- 
tudes which are more compatible with the work of a design en- 
gineer. It was with these subtle differences between the sales 
engineers and design éngineers in mind, that the test was devised. 

(b) Directions for the Test.——The nature of the test, being 
composed partly of questions of a technical nature as well as of 
general information, makes it inadvisable to publish it. However, 
we publish here the directions just as they are given with the 
test. The directions include samples which indicate the nature of 
the various kinds of problems in the test. These samples are to 
explain the test so that the subject will understand clearly what he 
is to do with each kind of problem; therefore, they are simple and 
easier than the problems and questions in the actual test. Never- 
theless, they sufficiently indicate the nature of this test for the 
purposes of this report. 
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TEST to 


TEST FoR GRADUATE STUDENT ENGINEERS 


Prepared by B. V. Moore, Research Fellow, Expressly for the Westinghouse 
Electric & Manufacturing Company, East Pittsburgh, Pennsylvania. 


Part I 


To the Westinghouse Student Engineer: 


This is a test to help you place yourself in the Westinghouse Company 
where you can be most successful. The results of the test will be held 
strictly confidential. You are asked to do your best with the test so 
that the Company will be in a position to give you the greatest possible 
opportunity. After the test has been scored, you may receive your score 
individually; and you may discuss the results confidentially with the 
executives of the Educational Department. 


Fill in the blanks as indicated. 


PN re ik eG ud bos ob ak OAD ECLe 6 Fan bee CUR oN U Raioe oss 

NE I he a a ie’ 6 od oo SARE M 0-6 ee SEAT OB bo do wees 

If not born in U. S., how many years have you been in U. S.?..........., 

Kind of Engineering for which you are trained..........c.ccecececceees. 

is (Eletrical, Mechanical, etc.) 
BOW & 05 bw eee OS 6 VK wh 6607500 008045 


Now read the Directions on the next two pages very carefully, but do 
not turn to page 4 until told to do so. 


DIRECTIONS 


Inside this booklet you will find a lot of things to do. Samples of all 
the different kinds of things to be done are given below with directions 
for doing each one. Follow the directions carefully, and pay close atten- 
tion to the.examples so that you learn to do each thing correctly. 


(a) People hear with the eyes ears nose mouth 
In such sentences, one of the last four words will make the 
sentence a true statement of fact. Underline the right word, thus: 
People hear with the eyes ears nose mouth 


Another example of the same thing correctly marked, is: 
France is in Europe Asia Africa Australia 





(b) locomotive—train :: horse— hub buggy car baggage 
The first word, “locomotive” is related to the second word, 
“train,” in the same way as the third word, “horse,” is related to 
one of the words following it. You are to underline that word 
which is related to the third word in the same way as the first two 
words are related to each other. In this example, “locomotive” 
is related to “train” as “horse” is related to “buggy”; for a locomo- 
tive pulls a train, and a horse pulls a buggy. Therefore, “buggy” 
should be underlined, thus: 
locomotive—train :: horse— hub buggy car baggage 


Another example of the same thing correctly marked, is: 
woman—beautiful :: man— girl old mother handsome 





Handsome is underlined because a man is described as hand- 
some in the same way as a woman is described as beautiful. 
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(c) The concert was (delicious lovely delightful) 
In such sentences, only one of the words or phrases in paren- 
thesis can be used correctly. You are to underline the correct 
word, thus: 


The concert was (delicious lovely delightful) 


Another example of the same thing correctly marked, is: 
He (isn’t doesn’t don’t) do that kind of work. 





(d) littl same as large child big small 
In this kind of problem, there is always just one word in the 
group of four words which has the same meaning as the single 
word given first. In this example, “little” has the same meaning as 
“small.” Therefore, “small” should be underlined, thus: 
little same as large child big small 


Another example of the same thing correctly marked, is: 
talk same as story truth speak hear 





(e) soft opposite of brittle hard yielding rough 


In this kind of problem, there is always just one word in the 
group of four words which has exactly the opposite meaning of the 
word given first. In this example, “soft” has the opposite 
meaning of “hard.” Therefore, “hard” should be underlined, thus: 
soft opposite of brittle hard yielding rough 


Another example of the same thing correctly marked, is: 
encourage opposite of courage inconvenience discourage inspire 





(f) Don’t put all your eggs in one basket. 
....The mouse that has only one hole is soon caught. 
....Catch the bear before you sell his skin. 
..+. Lhe proof of the pudding is in the eating. 


In such groups of statements, there is always just one which 
gives essentially the same meaning as the first one. Put a cross 
mere the statement which means the same as the first statement, 
thus: 

Don’t put all your eggs in one basket. 

.X.The mouse that has only one hole is soon caught. 
....-Catch the bear before you sell his skin. 

.... the proof of the pudding is in the eating. 

Another example of the same thing correctly marked, is: 

Long absent, soon forgotten. 

....Absence makes the heart grow fonder. 
....Distance lends enchantment to the view. 
.X .Far from eyes, far from heart. 


When the signal is given, you are to turn to page 4 and do these 
things as you have learned to do them here. Do not ask questions. If 
you forget how to do any of them, you may turn back to these pages, 
but to do so unnecessarily will waste your time. 


Work rapidly. You will be allowed twenty minutes. You may not 
be able to get through in that time; but do as much as possible. 


Mark every question. If you are not sure about your answer to 
something, guess at it, and go on to the next question. There are no 
catch questions. 

When the signal is given, begin at the top of the next page, and 
work through the remainder of the booklet, without skipping about. 


Do not turn over to the next page until you are told to do so. 
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Part II 


On the following pages are some more things for you to do. 
Samples of the three different kinds of things to be done are given below 
with directions for doing each one. Follow the directions carefully, and 
pay close attention to the examples so that you learn how to do each thing 
correctly. 

(a) silver copper glass aluminum gold 

In such groups of five words, there are four words which rep- 
resent things or ideas that can be thought of as similar in a certain 
way. That is, they can be classed under the same fundamental 
idea or conception. One of the five words represents something 
that is not like the other four words in the way that they are all like 
each other; and therefore, it cannot be classed with them under the 
same conception or category. You are to discover the fundamental 
idea of the classification, and underline the word that is not 
included. In the example above, all the materials are conductors of 
electricity, except “glass,” which is relatively a non-conductor. 

Therefore “glass” should be underlined, thus: 

silver copper glass aluminum _— gold 


Another example of the same thing correctly marked, is: 
shunt motor ___ transformer telegraph .sounder 





synchronous alternater exciter electroplating apparatus 
Transformer is underlined because it has to do with alternating 


current, but the other four words have to do with direct current. 


(b) Other factors remaining constant, the electric current in a wire 


varies inversely as the resistance of the wire. 
True False 


You are to decide whether such statements are true or false, and 
then underline the appropriate word, True or False, which indi- 
cates the nature of the statement. The above statement is true; 
therefore, “True” should be underlined, thus: 

Other factors remaining constant, the electric current in a wire 


varies inversely as the resistance of the wire. 
True False 


Another example of the same thing correctly marked, is: | 
Laminated armature cores are used, because they retain magnetism 
better. 








True False 


(c) What direct current at 110 volts will give the same horsepower as a 
direct current of 5 amperes at 220 volts? 


PEE ss 6 an vanes amperes 
Such problems are to be solved. You may use the margin of 
MY, the pages for figuring. Write the answer in the place indicated, 


thus: 
What direct current at 110 volts will give the same horsepower as a 


direct current of 5 amperes at 220 volts? 
Answer....10....amperes 


Another example correctly solved, is: 


A direct current motor uses a current of 7.46 amperes at 250 volts. 
A Pony brake test shows that the motor is giving 2 horsepower. 


What is the efficiency of the motor? 
Answer....80%.... 
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When the signal is given, you are to turn to the next page and 
do these things as you have learned to do them here. Do not ask ques- 
tions. If you forget how to do any of them, you may turn back to 
these pages, but to do so unnecessarily will waste your time. 


Work rapidly. You will be allowed thirty minutes. You may not 
be able to get through in that time, but do as much as possible. 


Mark every question. If you are not sure about your answer to 
something, guess at it, and go on to the next thing. There are no catch 
questions. 


When the signal is given, begin at the top of the next page, and 
work through the remainder of the booklet, without skipping about. 


Do not turn over the page until you are told to do so. 


(c) Construction of the Test.—A test is developed and im- 
proved much as a new machine is developed. It is brought nearer 
perfection, and the final efficiency attained by eliminating one 
difficulty after another and by taking advantage of one pos- 
sibility for improvement after another, just as the dynamo was 
brought nearer perfection from a cast iron bipolar series-wound 
machine to the present efficient machine with laminated cores, 
commutating poles, commutating pole-face windings, compound 
wound field, balance coils, etc. This test was developed in much 
the same way. It may be found later that some of the reasons 
for making the test as it is, have no basis of fact; but the inten- 
tion was not to fail to take advantage of any possible means or 
device for differentiating between the sales engineer and the de- 
sign engineer type. Therefore, some of the features of the test 
may seem trivial and even arbitrary; but they were not discarded, 
because there was no proof of their futility. 

To discuss the test more in detail, we shall begin with the in- 
formation questions of Part I. (See Directions, a). Of the one 
hundred sixty questions in Part I, forty, or one-fourth, are ques- 
tions of information. It is believed that the sales type of engineer 
has been interested in other things in addition to his books and 
strictly technical training. He not only has been a good mixer 
in college and has taken part in extra-academic activities, but he 
also has been interested in the affairs of the social, economic, busi- 
ness, and work-a-day world. The potential salesman’s abilities, 
and with them his interests, have tended to select and retain for 
him those items of information which are peculiar to a sales type 
of person and make him such a person. A definite method was 
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followed in order to have the questions such that a sales type of 
person would answer best. After studying the occupational 
descriptions of the various lines entered by sales engineers, a 
definite prospective customer in a particular industry was 
imagined. Supposing that I was sales manager, I outlined the 
kind of experiences and information I should want my salesman 
going to that customer to have. After doing this for each class 
of customer for each sales department, I prepared a composite list 
of kinds of information and experience which I should expect a 
successful sales type of engineer to have accumulated in his 
early life, his high school and college ‘life, his social and fra- 
ternal life, his business life, his sports, amusements, and avoca- 
tions, and his reading. With this list before me, I selected from 
other tests and formulated new questions which would cover 
these twenty or more different classes of information and inter- 
ests. 

The second type of problem chosen was the analogies test. 
(See Directions, b). Forty items of this type were included. In 
studying the general intelligence test papers, Bureau Test VI, 
the scores of the sales engineers in the different kinds of tests 
making up the total test were compared with the scores in those 
tests of the other engineers making the same total score. It was 
found that with the small number considered, twenty-five sales 
engineers and sixteen others, the sales engineers did better in 
the analogies test, although their total score on the whole exam- 
ination was the same as the other engineers’ with whom they were 
compared. 

The next three tests might be considered or classified as 
vocabulary tests, but they are intended to be also more than that. 
Recognizing the fact that the salesman must be able to express 
his ideas and understand what others are trying to say, an at- 
tempt is made to measure not merely his knowledge of words, 
but also his sense of their finer meanings and implications in 
certain usages. The correct answer to a particular test item of 
this sort often cannot be made according to any dictionary defini- 
tion or any rule of rhetoric. It depends upon usage and a fine 
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sense of propriety. To answer them correctly the man cannot 
stop to reason out the answer, or he will be lost; but he must 
respond as he feels about the proper sense of the word. It is an | 
attempt to measure the ability not of using words but of compre- 
hending the meaning of other people’s statements and of getting 
other people to undertsand us as we expect them to understand 
us. 

The proverbs are intended to measure somewhat the same thing 
as the preceding types of tests measure, namely ability to read 
and understand sentences. In addition, it is considered as a test 
of the ability to generalize or think consistently. It measures 
the ability to interpret the gist of a complete statement rather than 
the meaning of words or phrases. In the proverbs test, several 
possible forms were considered. I thought at first that Professor 
L. L. Thurstone’s form, a prototype with four other sentences, 
would be used; but this was abandoned as too long for one item. 
Then I thought that it would be possible to score Professor 
Thurstone’s form as two items, one point for each sentence or 
phrase checked correctly. But considering the probabilities in- 
volved, this plan was dropped. By that plan, a subject in.select- 
ing the first of the two sentences to be checked, has a two-out-of- 
four or fifty-fifty chance. If he is a capable subject he gets the 
first one right. Then in checking the second one he has a chance 
of only one out of three; and thus the problem is made more 
difficult. However, if he were a much less capable subject, the 
first one he checks might very possibly be a wrong one; but he has 
two chances out of three of selecting the right one in checking a 
right sentence as the second one. Thus the problem is made 
more difficult for the more capable and less difficult for the less 
capable ; and there is a tendency to eliminate a showing of differ- 
entiation in capacity. For these reasons, the proverbs were con- 
structed so that there would be just one probability, namely one 
out of three chances. 

Part II. is apparently a test of technical engineering ability. 
However, it is not to be considered a trade test. It would be 
such if it were given to a group of men, some of whom were 
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engineers and some of whom were not, but the test is for en- 
gineers only, and is not to distinguish engineers from people 
that are not engineers. The technical information required to 
pass the test is comparatively easy for electrical engineers; for 
the test was designed to require only such information and en- 
gineering ability as every graduate of all standard schools would 
have. When the test is given to a group of engineers, it is essen- 
tially an intelligence test ; but it tests a certain kind of intelligence. 
The hypothesis upon which it is based is that the man who is 
more capable as an engineer, who has his interests in the pro- 
fession, and who has held himself more intensively to his books, 
shop, drafting board, and mathematics, is the design engineer 
type, and is the kind of engineer who will tend to pass this kind 
of test better than he would pass Part I. It is expected that a 
pure design engineer type of man will pass such an intelligence test 
better if the vocabulary and content is that with which he is ac- 
customed to work and think about. 

The first kind of problem or question in Part II, (a), is, as the 
Directions explain, a test of ability to deal with fundamental con- 
ceptions found in mechanics, physics, or engineering. The test 
is intended to measure the subject’s alertness in dealing with or in 
juggling such technical ideas. It is even hoped to be a test of 
association or originality in seeing the relations involved in such 
fundamental conceptions. It calls for analysis and generaliza- 
tion in the technical field. 

The second type of question, Part II, Directions (b), is com- 
posed largely of questions calling for technical information; but 
the questions are so stated that the subject must read them care- 
fully and think straight to give the right answer; for they can- 
not be answered directly from general principles or textbook 
information, but require the application of general principles. 

The third type of problem, Part II, Directions, (c), is mani- 
festly a straight mathematical engineering problem. The prob- 
lems are not highly technical, but require straight thinking and 
accurate computation. Problems in mensuration, mechanics, 
and general physics are included as well as problems in electrical 
engineering. 
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In a psychological test with a time limit, it seems most logical 
to have the steps or items to be solved, of equal difficulty, par- 
ticularly if speed is considered as a measurable factor of the 
ability to be tested. Therefore, if the items cannot be made of 
equal difficulty, it does not seem consistent to arrange the items 
in the order of their difficulty, putting the most difficult to be 
solved last. This tends to minimize the differentiating process 
rather than make the most of it. The capable subject gets 
through the easier items first, but is held back by the more difficult 
material ahead, while the less capable subject coming through 
the easier material then tends to catch up with the capable man. 
Moreover, the difficult material which the capable person might 
be able to solve, but. which the less capable man never could 
solve, is never reached by either. It might be argued that the 
more difficult material be placed first and the easier later, so that 
the less capable subjects would have to pass it over after spend- 
ing a little time on it. The capable man would get credit for it 
and also for much more easier material, which would make his 
score much larger than that received by the less capable man. 
Thus the differentiation would be greater. For these reasons, no 
especial effort was made to rank the problems, particularly in 
Part II, in order of difficulty; but some attempt was made to 
keep the difficulty fairly constant. Of course the first few items 
were made easier so that all subjects could get the correct idea 
and get fairly started. 

(d) Standardization of the Test—In the process of develop- 
ing and testing the test, Part I. was given first to the members of 
the seminar of the Bureau of Personnel Research, composed of 
two persons with the degree of Doctor of Philosophy in psycholo- 
gy, and six graduate students. The number of the item which 
was reached by each subject at the end of twenty minutes, was 
checked, and the subjects asked to complete all the items of the 
test. Also, the subjects in the seminar were asked to criticize 
and put a question mark after each item that was ambiguous or of 
questionable reliability. Then both the errors and the criticisms 
were tabulated for each of the one hundred sixty items. After 
making a few minor changes in Part I. as a result of this first 
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trial, it was given to a class in the School of Insurance Sales- 
mahship. Sixty-one men and three women, about half of whom 
were college people, were in this group taking the test. Part I. 
was then revised a second time and given to fifty-five men and 
five women in another class in the School of Insurance Sales- 
manship. 

The process of constructing and standardizing Part II. was 
somewhat different. The True-False questions were taken from 
Professor E. L. Thorndike’s engineering test. The problems 
were taken partly from Professor L. L. Thurstone’s engineering 
tests, partly from standard engineering textbooks, and part 
were originally devised by myself. The disparate word test items, 
that is, the groups of words with one word not included in the 
fundamental conception, were all originally devised by myself. 

For standardizing Part II., it was studied and the problems 


‘worked by two engineers of the Westinghouse Electric and 


Manufacturing Company. Then the problems were worked, and 
the whole of. Part II. carefully criticized by Professor W. R. 
Work of the Department of Electrical Engineering, Carnegie 
Institute of Technology. Then both Part I. and Part II. were 
given to three seniors in electrical engineering just graduating 
from the Carnegie Institute of Technology. After a third re- 
vision, the entire test, Part I. and Part II., was given to twenty- 
eight design engineers and two general engineers in the Westing- 
house Electric and Manufacturing Company. Later the entire 
test was given to fifty-nine sales engineers in the same company. 

The specifications for selecting the Westinghouse engineers for 
the test were: (1) They all should be graduates of technical 
schools. (2) They should have been out of school and in actual 
work with the Company not more than five years and not less 
than one year. (3) The design engineers should be beyond 
doubt of the type desired in design engineering. They should 
have proven successful and should be satisfied in their work. 
There should be no doubt that they were properly placed. (4) 
The sales engineers should be beyond doubt of the type desired 
in the sales department. They should have proven successful 
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and should be satisfied in their work. There should be no doubt 
that they were properly placed as sales engineers. 

With these specifications, the heads of the engineering depart- 
ments selected the design engineers; and the whole list was gone 
over carefully by the Chief Engineer. The sales engineers were 
selected in a similar way by the sales managers. A few of the 
sales engineers had been out of school more than five years; but it 
is believed that the two groups were composed of as pure type 
representatives of the two kinds of engineers as could be ob- 
tained. In addition to helping make the selection, the Chief 
Engineer rated the design engineers on their ability as design 
engineers; and the sales managers rated the sales engineers on 
their ability as sales engineers. 

(e) Results of the “Test for Graduate Student Engineers.”— 
The results of the test given to the different groups can be sum- 
marized briefly. Because the test was revised after giving it to 
the first class in the School of Insurance Salesmanship, the scores 
can not be compared with the scores of the other groups. The 
insurance salesmanship classes had also taken Bureau Test VI., 
and the correlations between the scores of that test and those of 
Part I, were, r == +-.64 for the first class, which took the original 
form of the test; and r == + .g2 for the second class, which took 
the revised form of the test. Of the engineers who took Bureau 
Test VI, 18 also took the Bureau Test 10. For these 18, the 
correlation between Test VI and Part I of Test 1o was r = + -.69; 
and between Test VI and Part II, the correlation was r = -+-.38. 
The correlation between the committee’s original estimate of in- 
telligence and Part I was r = +-.49; and between estimated in- 
telligence and Part II, the correlation was r == +.76. Evidently 
the estimate of intelligence was influenced by the student engin- 
eer’s manifestation of engineering capacity. 

Considering only the design engineers, Diagrams 12, 13, and 
14 are four-fold tables showing the relation between the results 
of Part I and the Chief Engineer’s ratings, between Part II and 
these ratings, and between a composite of Part I and Part II and 
these ratings. The composite score was made by adding the 
score in Part I to the score in Part II in terms of standard devia- 
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Diagram 12. Comparison of 
Scores in Part I with Chief En- 
gineer’s Rating of Design Engi- 
neers. fy = .22, 
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Diagram 14. Comparison of 
Scores in Composite of Part I 
and Part II with Chief Engineer's 
— of Design Engineers. 
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Diagram 16. Comparison of 
Scores in Part II with Sales 
Managers’ Ratings of Sales En- 
gineers. ry = + 81. 
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Scores in Part II with Chief En- 
gineer’s Rating of Design Engi- 
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Diagram _ 15. Comparison of 
Scores in Part I with Sales Man- 
agers’ Ratings of Sales Engineers. 
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Composite Scores, Parts I and II 


Diagram 17. Comparison of 
Scores in Composite Of Part I 
and Part II with Sales Man- 
agers’ Ratings of Sales Engl- 
neers. fy = + 66. 
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tions. It is evident that Part II, the technical part prepared for 
design engineers, correlates more nearly perfectly with the rat- 
ings than Part I does. The composite score, which takes into 
account both the special engineering capacity and the general in- 
telligence or mental alertness of the man, correlates a little more 
nearly perfectly. Diagrams 15, 16, and 17 show the same 
relations for the sales engineers; that is, the correlations between 
the test and the ratings made by the sales managers. 

Considering again only the design engineers, the mean of the 
scores in Part I was 109.3; and the mean of the scores in Part 
II was 37.0, which is approximately one-third of the mean for 
Part I, 37.0 multiplied by three being 111.0. For the sales en- 
gineers, the mean of the scores in Part I was 87.6; and the mean 
of the scores in Part II was 22.7, which is approximately one- 
fourth of the mean for Part I, 22.7 multiplied by four being 
90.8. Thus the sales engineers do relatively better in Part I as 
compared to Part II than the design engineers do. Comparing 
in another way, the design engineers do relatively better in Part 
II as compared to Part I than the sales engineers do; for the 
mean of their scores in Part II is more than one-third of the mean 
of the scores in Part I. 

Diagram 18 is a scatter diagram showing the relation between 
Part I. and Part II. for the scores of the design engineers and 
the sales engineers. Diagram 19 shows the same relation between 
the percentiles of each man in Part I and Part IJ. The circles 
(OQ) represent the sales engineers; and the crosses (X) represent 
the design engineers. One purpose of the test is to distinguish 
sales engineers from design engineers. According to the hypo- 
thesis for the test, the men who did well in both parts of the 
test and appear in the upper right-hand quadrant of the diagram, 
are mentally capable of being either sales engineers or design 
engineers because of their superior general all-around intelligence, 
or because of their superior intelligence in at least these two fields. 
The men who did poorly in Part II. but very good in Part I, 
should be those whose kind of intelligence fits them better for 
the work of a sales engineer. The larger percentage of circles 
in this quadrant substantiates this part of the hypothesis. Those 
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Scores in Scores oF Part II (TECHNICAL) 
Part I. 0-3 4-7 8-11 12-15 16-19 20-23 24-27 28-31 32-35 36-39 40-43 44-47 48-51 52-55 56.60 
(General) 
15 5-160 
150-154 
145-149 
140-144 
135-139 
130-134 
125-129 
120-124 
115-119 
110-114 
105-109 
100-104 
95-99 
90-94 
85-89 
80-84 
75-79 
70-74 
65-69 
60-74 
55-59 
50-54 
45-49 
40-44 
35-39 
30-34 
25-29 
20-24 
15-19 
10-14 
5-9 
0-4 





Diagram 18. Scatter Diagram Showing Relation of Scores in Part I 
to aay in Part II for Design Engineers (+) and for Sales Engi- 
neers (0). 


who did poorly in Part I. but very good in Part II., should be 
those whose kind of intelligence fits them better for the work 
of a design engineer. The larger percentage of crosses appear- 
ing in this quadrant substantiates this part of the hypothesis. 
Those doing poorly in both Part I. and Part II., might be equally 
successful in either line of work, as far as mental ability is con- 
cerned; but they are not as successful as any of the others might 
be if properly placed in the right line of work. As a whole the 
hypothesis is substantiated by the results. There are some 
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Percentiles PERCENTILES IN Part II (TECHNICAL) 
in Part I. 0-9 10-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90-100 
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Diagram 19. Scatter Diagram Showing Relation of Scores in Part I 
to Scores in Part If in Terms of Percentiles for Design Engineers (+) 
and for Sales Engineers (0). 
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exceptions as seen from the positions of the circles and crosses 
on the scatter diagram; but investigation shows reasonable causes 
for these, which do not disprove the hypothesis. The test was 
made for graduate student engineers in the Educational Depart- 
ment of the Westinghouse Company, and all these engineers are 
always just graduated from the technical schools. Therefore, 
they would not have the difficulty of having specialized in in- 
dustry for several years and having forgotten some of their theor- 
etical or technical training. Some of the design engineers who 
took this test had been doing very highly specialized work for 
five years, and had forgotten some of the information in other 
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lines of electrical engineering. For this reason, they may not 
have done so well in Part II, although they were regarded by the 
Chief Engineer as among his best engineers, particularly because 
of their experience, conscientious work, and constant applica- 
tion. However, I do not think that the specialization or ex- 
perience of the design engineers nor of the sales engineers ma- 
terially affected their scores; for Part II is a test of engineering 
aptitude and not of engineering information, and Part I is a 
general intelligence or mental alertness test. The correlation 
between Part I and Part II was r = +-.42 for the design en- 
gineers, and r == -++.84 for the sales engineers. If the design 
engineers have special capacity for engineering, as we believe 
they have, it is to be expected that their score in Part II, which 
tests engineering capacity, would correlate less with their scores 
in Part I, which is a general intelligence test, than the sales en- 
gineers’ scores in Part II would correlate with their scores in 
Part I. The scatter diagram of the scores of the design en- 
gineers in Part II shows that the correlation between the parts is 
lower because some of the design engineers had special aptitude in 
engineering, but could not do well in the general intelligence or 
mental alertness test. 

(f) Statistical Interpretation of the Results for Occupational 
Placement.—The test is to be used in two different ways for two 
different purposes. The first use is to separate the men who are 
best fitted to be sales engineers from those who are best fitted 
to be design engineers. What is desired in this case is to know 
for what the men are best fitted by their special abilities, not con- 
sidering their general intelligence or mental alertness. A man 
might be more successful as a design engineer and might make 
a better score in Part II than another man; but even so he would 
not be so well placed if he is placed as a design engineer when 
he could be still more successful as a sales engineer. Each man 
should be placed not where he would be merely successful, but 
where he could function most efficiently and be most successful. 
Therefore a single score in a sales test or in an engineering test 
could not be relied upon to place the men; for the good all-around 
man would do best in both kinds of work, and best in both kinds 
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of tests. However, some measure of the kinds of abilities or 
the qualitative differences in intelligence could be obtained by 
subtracting the score of each man in one test from his score in 
the other test. Suppose, for each individual, we subtracted the 
score of Part II in terms of the standard deviation from the 
score of Part I in terms of the standard deviation. The man 
who had a score in Part I which was considerably higher than 
his score in Part II would have a large algebraically positive dif- 
ference between his scores, and he would be a sales engineer type 
of man. This great difference between his scores in each part of 
the test might be because he did well in Part I, or did poorly in 
Part II, or for both reasons. In any case, he would be better 
fitted for sales engineering than he would be adapted for design 
engineering. The man who had a score in Part I considerably 
lower than his score in Part II, would have a large algebraically 
negative difference between his scores; and he would be a design 
engineer type of man. This great difference between his scores 
in each part of the test might be because he did poorly in Part I, 
or did well in Part II, or for both causes. By use of this method, 
that is, transmuting the scores into terms of sigmas and sub- 
tracting the score in Part II from the score in Part I, those who 
get the greater algebraically positive differences are to be con- 
sidered the more specially fitted for sales engineering ; and those 
who get the greater negative differences are to be considered the 
more specially fitted for design engineering. The practical use 
of this method is to rank the men according to the algebraical 
differences between their respective scores in Part I and Part II, 
and then count down the list until the desired number of sales 
engineers is obtained. That is, the man who did well in Part I, 
but poorly in part II, would rank high in the list according to the 
algebraic difference between his score in Part I and his score in 
Part II, and he would be chosen for sales engineering. This 
method, theoretically, selects the men who can function most 
efficiently as sales engineers, and leaves those who can function 
most efficiently as design engineers. 

To check this method of classifying and ranking the thirty 
design engineers and the fifty-nine sales engineers, the score of 





‘ep, 





Diagram 20. 
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each individual for Part II was subtracted from his score in Part 
I, both scores being in terms of their respective standard devia- 
tions. This gave positive although not the most desirable results 
in separating the sales engineers from the design engineers. 
These results are shown graphically in Diagram 20. Consider- 
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ing the diagram as a four-fold table, this method of interpreting 
the results differentiates the men so that of those classed as sales 
engineers, 79 per cent are correctly so classed; and out of those 
classed as design engineers, 49 per cent are correctly so classed. 
The correlation between the test results treated in this way and 
the actual classification was ry == + .45. Also, the correlation 
between the classification of the men by the test treated in this 
way and the classification of the men by the “Section II. Choice 
of Other Occupations” of the Record of Interests, was r== +.50. 
A similar and simpler method of treating the results is to trans- 
mute the raw scores into percentile ranks, and subtract the per- 
centile rank in Part II from the percentile rank in Part I. This 
was done, and the classification of the engineers was practically 
the same; for the correlation between differences in percentile 
ranks (Part I minus Part II and the differences in scores in terms 


Differentiation Obtained by Subtracting Scores in Part I from 
Scores in Part I in Terms of Standard Deviations. 

Considering the diagram as a four-fold table, the engineers indicated by the test 
as sales engineers are 79 per cent correctly so classified; and the engineers indi- 
cated by the test as design engineers are 49 per cent correctly so classified. 


ry = + -45. 
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of sigma (Part I minus Part II in terms of sigmas) was 
r= +.97. 

The best method of dealing with the results is to divide the 
score in Part I by the score in Part II for each individual. This 
is the method which is the simplest, and it is the one which gives 
the most efficient use of the test as a means of differentiating 
the men. This method magnifies the differences among the in- 
dividuals, which differences are shown by showing that the ability 
of each individual to deal with Part I differs from his ability 
to deal with Part II. That is, the ratio or quotient of a man’s 
score in Part I divided by his score in Part II changes geo- 
metrically, instead of algebraically, as this man’s special ability 
is greater or less. This probably makes the results agree more 
nearly with the practical considerations; for the extreme cases of 
special ability should be markedly differentiated. The genius 
is valued probably far greater than an actual measurement of 
capacity would indicate. The results of dealing with the scores 
by this method are presented graphically in Diagram 21. Consid- 
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aera 21. Differentiation Obtained by Dividing Scores in Part I by Scores 
in Part II. 

Considering the diagram as a four-fold table, the engineers indicated by the test 
as sales engineers are 81 per cent correctly so classified; and the engineers indi- 
cated by the test as design engineers are 63 per cent correctly so classified. 


Ty = +.70. 








74 BRUCE V. MOORE 


ering the diagram as a four-fold table, we have the engineers 
differentiated as follows: | 

Knowing first the actual classification of the engineers, we can 
say that 63 per cent of the design engineers were shown by the 
test to be better fitted for design engineering than 81 per cent 
of the sales engineers were shown to be. 81 per cent of the 
sales engineers were shown by the test to be better fitted for sales 
engineering than 63 per cent of the design engineers were shown 
to be. Not knowing first the classification of a group of en- 
gineers, this test can be used to differentiate them as follows: 
Assuming that if an engineer’s score in Part I of the test is more 
than 3.2 times his score in Part II, he is a sales engineer, and 
that if his score in Part I is less than 3.2 times his score in Part 
II, he is a design engineer or a pure type of engineer, this test 
divides the engineers so that 81 per cent of those above this crit- 
ical ratio (Part I at least as much as 3.2 times Part IT) are sales 
engineers ; and 63 per cent of those below it are design engineers 
or of a purely engineering type. (This critical ratio may be 
changed slightly by the change from the mimeographed form of 
the test to the printed form.) The correlation between the test 
results treated in this way and the actual classification of the 
men were ru == + ..70. Also, the correlation between the classifi- 
cation of the men by the test results treated in this way and the 
classification of the men by the Record of Interests, Section II, 
was r= +.46. The correlation between the marks for the en- 
gineers when the scores were treated in this way (score in Part 
I divided by score in Part II) and the marks for the engineers 
when the score in part II was subtracted from the score in Part 
I in terms of sigmas, was r == +.50. Such was the correlation 
coefficient obtained when the quotients (Part I divided by Part 
IIT) were correlated with differences between scores (Part I 
minus Part II in terms of sigmas) by the Pearson product-mo- 
ment formula. However, this does not show the true relation- 
ship; for the relationship is a non-linear one; and by using the 
formula for eta, it is shown to be y — .88. 
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The second use of the test is to rank the sales engineers and 
the design engineers after they have been separated, so as to show 
who are the best design engineers, and who are the best sales 
engineers, considering general intelligence or all-around mental 
ability as well as special ability. In other words, this use of the 
test makes it a general intelligence or mental alertness test. The 
relationships of the results of the test with the estimates of the 
success Of the engineers have already been given in the pre- 
vious section. There the composite of Part I and Part II 
was obtained by simply adding the scores of the two parts in 
terms of their standard deviations. The composite does not 
necessarily show a higher correlation with the estimates of suc- 
cess than each part separately shows; for if the two parts test 
two widely different abilities, then combination may not give a 
higher correlation with the criterion. The simplest method of 
combining the scores is to add the percentile rank of each man in 
Part I to his percentile rank in Part II, and then rank the men 
according ‘to these sums of percentile ranks. This gives results 
which are practically as reliable as those obtained by the method 
of combining scores in terms of sigmas. 

On the basis of the results obtained so far, the test, both Part 
I and Part II, is being given to the graduate engineers who are 
entering the employ of the particular industrial firm. Only by 
following up these men in the lines of work which they enter 
later, will we finally know the value of the rating scales, of the 
Record of Interests, and of this special aptitude test. 

The results already obtained on one hundred seven engineers 
employed by the firm during the first year that the test has been 
used are very similar to the results obtained on the group studied 
in this research. These hundred and seven men have been with 
the company approximately one year and they have been classi- 
fied according to previous methods into sales engineers and de- 
sign engineers by the executives supervising their work. The 
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test results were not a factor in the classification. However, 
assuming that the executives classified the men correctly at the 
end of a year, we can say that the test results alone would have 
classified seventy per cent of the men correctly on the day that 
they were employed. Beginning with June, 1921, the company 
will use the test and other methods recommended here as definite 
aids in classifying and placing the graduate engineers whom they 
employ. 


PART III. SUMMARY OF RESULTS AND CONCLD- 
SIONS AND RECOMMENDATIONS 


X. SuMMARY OF RESULTS 


The practical problem upon which we began work was to 
formulate methods and means based upon sound psychological 
principles for differentiating graduate engineers. The results 
of studying various existing methods and certain proposed or 
newly devised ones showed that some were of value and that 
others were not. The results might be summarized briefly as 
follows: 

1. Technical school grades when coming from _ various 
schools of different standards, do not differentiate the men in a 
large group according to their ability in industry, nor according 
to the kind of work which they can do best. This is with the 
one exception that men who are selected to be design engineers are 
the kind of men who made higher grades in technical schools. 
This was true in all subjects except in general academic subjects, 
particularly economics, in which the design engineers fell below 
their own average, and the grades of the sales engineers were 
above their own average, so that the grades of the two groups 
were practically the same in these subjects. 

2. Neither the grades given in the classes of the Educational 
Department nor the grades given in the shop on the courses of 
the Educational Department show any consistency or reliability. 
The correlation between the two sets of grades is only r= +.19, 
and the correlation is zero with all other measures such as tech- 
nical school grades, ratings, and test scores. The unreliability of 
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these grades and ratings is doubtless due to the method of making 
the ratings, of recording them, and of combining them, rather 
than being altogether due to the inability of the foremen to make 
the ratings. 

3. The ratings made by the foremen on the ten different traits 
or qualifications show no agreement with any other measures 
such as shop and class grades, ratings, technical school grades, or 
test scores. There is high intercorrelation among the traits. 
This is due to two causes: (a) Some of the traits are very sim- 
ilar, or at least they are not well enough defined for the foremen 
to distinguish them; and as the foremen are not trained to make 
such ratings, they decide that a man is either a good man or a 
bad man, and then rate the man accordingly either good or bad 
in all traits. (b) The method of making the ratings, in terms of 
A, B, or C, does not give the foremen any scale or basis of 
measure or comparison. 

4. The only traits rated by the foremen which differentiate 
the different groups of graduate student engineers, are tact, 
initiative, enthusiasm, and personality. The trait called per- 
sonality is the most vague and undefined, but it shows the greatest 
differentiation of the men; for it is a trait concerning which the 
foremen really get an impression and form an opinion in regard 
to each man. 

5. The Interviewer’s Rating Scale as already in use by the 
Company, has given results that have significant correlation with 
test scores, with later estimate of intelligence, and with technical 
school grades. The trait called Personal Qualities, as previously 
defined, had a high correlation with other traits, and was too 
broad, inclusive, and ambiguous to give reliable results. It was 
then more narrowly defined so as not to be ambiguous and not 
overlap or include the other traits to be rated. 

6. The College Instructor's Rating Scale was prepared to en- 
able the college instructors, professors, or deans to record a 
definite estimate of a senior on specific traits. This is expected 
to give more reliable and significant ratings of a man than a 
verbal expression of opinion of the man in general. 

7. The Shop and Class Rating Scale was prepared to take 
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the place of the method of rating on ten traits by A, B, and C. 
It requires ratings on a few very significant traits or qualities ; but 
it is expected to enable the foremen and the instructor to have a 
common understanding of just what is meant by these traits and 
to give a reliable and significant rating that will differentiate the 
engineers according to essential qualifications. 

8. The use of the Record of Interests shows that men in dif- 
ferent lines of work have different interests. That is, the insur- 
ance Salesmen as a group, have always had interests different 
from either sales engineers or design engineers. The sales en- 
gineers are men who have been more interested in economic, 
public, and social affairs. The design engineers are men who 
have been more interested in science, machinery, and actual con- 
struction or fabrication of things. 

9. The engineers and the salesmen showed a definite tendency 
to like or be interested in occupations which in nature of work 
were similar to those they were already following. The kind of 
occupations which they thought would give them the greatest 
satisfaction to follow, was a significant criterion of the kind of 
work in which they could be and were already successful. 

By computing for each engineer the percentage which his 
choices of occupations of a sales nature were of his total number 
of choices, a definite measure of his sales engineering interest as 
opposed to design engineering interest was obtained. By this 
measure, 78 per cent of the sales engineers were of a sales type; 
and 82 per cent of the design engineers were of an engineering 
type. Or assuming that we did not know the actual occupations 
of the engineers, those engineers which this test of interests 
alone would select for sales engineering, would be 89 per cent 
correctly placed or classified; and the men which the test of in- 
terests selected for design engineering would be 68 per cent cor- 
rectly placed or classified. 

10. Personnel Bureau Test VI, a general intelligence test 
similar to the Army Alpha intelligence test, gave results that did 
not differentiate the engineers according to the kinds of work for 
which they were thought to be fitted. The only men who had 
scores significantly different from the other engineers, were the 
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design engineers. They were a highly selected group of men who 
had shown ability in the engineering classes of the Company’s 
Educational Department. | 

11. The general intelligence test gave scores that correlated, 
r == +.48, with intelligence as estimated by two executives of 
the Educational Department who were intimately associated with 
the graduate student engineers one year. 

12. The general intelligence scores correlated, r — +.38, 
with technical school grades, and with interview ratings at the 
time of employment, r= -+.58; but the correlations were zero 
with shop and class ratings, which did not correlate with any 
other criteria, and were evidently unreliable. 

13. Test 10, A Test for Graduate Student Engineers, gave 
positive although not perfect results in separating the sales en- 
gineers from the design engineers. Knowing first the actual 
classification of the engineers, we can say that 63 per cent of the 
design engineers were shown by the test to be better fitted for 
design éngineering than 81 per cent of the sales engineers were 
shown to be. 81 per cent of the sales engineers were shown by 
the test to be better fitted for sales engineering than 63 per cent of 
the design engineers were shown to be. Not knowing first the 
classification of a group of engineers, this test can be used to dif- 
ferentiate then as follows: Assuming that if an engineer’s score 
in Part I of the test is more than 3.2 times his score in Part II, 
he is a sales engineer, and that if his score in Part I is less than 
3.2 times his score in Part II, he is a design engineer or of a 
pure engineering type, this test divides the engineers so that 81 
per cent of those above this critical ratio (Part I 3.2 times Part 
IT) are sales engineers, and 63 per cent of those below it are de- 
sign engineers or of a purely engineering type. (This critical 
score may be changed slightly by a change from the mimeo- 
graphed form of the test to the printed form.) 

14. The differentiation of the engineers resulting from the 
use of Test 10 showed a comparatively high correlation with 
the results of the Record of Interests. The correlation was 
r == +.50 when the scores in Part II were subtracted from those 
in Part I in terms of sigmas. The correlation was r = +.46 
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when the differentiation was obtained by dividing the scores in 
Part I by the scores in Part II. This shows not only that there 
is high correlation between interest and special ability, but also 
that the Record of Interests and Test 10 give reliable results. 

15. Test 10 used as a general intelligence or mental alertness 
test for sales engineers gave scores that correlated, r == -++.86 
and -+.81 with estimated success. The same test used as a gen- 
eral intelligence test for design engineers gave scores that cor- 
related, r==-++.22 and +.59, with estimated success. 


XI. CONCLUSIONS AND GENERAL PRINCIPLES DERIVED 
FROM THE STUDY 


1. The study of the technical school grades show that such 
grades cannot be made of practical use to differentiate engineers 
for various kinds of work, except in a very general way. Sales 
engineers do comparatively better in economics than in engineer- 
ing subjects. The design engineers make much lower grades in 
economics than they do in mathematics and science, engineering, 
and shop. 

2. When foremen are asked to rate men under them on a 
large number of traits which are not carefully defined, the ratings 
are unreliable and of little use as a measure of the value of the 
men rated; and such ratings are particularly of no use for classi- 
fying the men and placing them in positions for which they are 
best fitted. Generalizing, we can say that ratings made by dif- 
ferent people on traits which are understood by some to be one 
thing and by others to be another, are of no value. This difficulty 
arises if the traits are not carefully defined in terms that the 
raters can understand; and there is the same difficulty if there 
are many traits to be rated, some of which overlap or include 
others so that they can not be differentiated. The ratings are un- 
reliable if there is no scale or standard with which the men can 
be compared. This basis of comparison may be the other men of 
the same group; but there must be definite comparison instead of 
arbitrary grades or marks. Finally, the ratings or marks must 
be recorded in such a form that they can be readily tabulated and 
actually used in placing the men. 
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4. Carefully prepared acquaintance-comparison rating scales, 
such as the Interviewer’s Rating Scale and the Shop and Class 
Rating Scale, give much more reliable results than forms on 
which arbitrary grades or marks are recorded opposite certain 
names of traits. The acquaintance-comparison rating scale en- 
ables the rater to compare the one to be rated with other people 
as concrete examples of different grades of the same trait. 

5. The use of the Record of Interests shows that men in dif- 
ferent lines of work have different interests. The interests of a 
person are not in just one specific occupation, but they are general 
to the extent that they pertain to very similar or closely allied oc- 
cupations or activities. Interest in a certain class of activities is 
a criterion that the person will be interested in any other very sim- 
ilar activity. By similar activity is meant one that requires much 
the same information, training, experience, kind of materials and 
tools worked with, mental activity, personality, ideals of accuracy 
and perfection, and social attitude. 

6. Inthe engineering profession, men cannot be differentiated 
for different lines of work by tests which measure what is com- 
monly known as general intelligence or mental alertness. Men 
in the same kind of engineering work differ in this intelligence 
as greatly as men in different kinds of engineering work. The 
differences which fit these men for different kinds of work, are 
something other than differences in intelligence. 

7. General intelligence as measured by a general intelligence 
test does show a significant positive correlation with the success 
of engineers in the same kind of work. 

8. One man differs from another in having special abilities, 
which functioning in an integrated form as a kind of intelligence, 
can be measured and used as criteria for placing the man in the 
kind of work for which he is best fitted. 

9. The occupational interests of a man show a definite cor- 
relation with the kind of intelligence or special abilities which he 
has, and with the kind of occupation in which he is successful. 
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XII. Specrric METHODS AND PRACTICES RECOMMENDED 


1. Before senior engineers who are being graduated from 
the technical schools, are employed, they should be rated by the 
use of rating scales. The representatives of the Company who 
interview the seniors, should use the /nterviewer’s Rating Scale. 
Before the interviewers start on their visits to the colleges, they 
should carefully make a Master Rating Scale from the list of 
engineers whom they know intimately and who have passed 
through the Educational Department within the previous five 
years. If possible, two or three interviewers representing the 
Company should interview the senior and make ratings independ- 
ently, and then combine the ratings later. 

2. After the interviewers have returned to the home office 
and have prepared a large tentative list of seniors who are to be 
considered for employment, the name of each senior should be 
put on a blank College Instructor's Rating Scale, and this blank 
sent to the dean, professor, or instructor who can be depended 
upon to give a reliable rating of the man under consideration. A 
rating by more than one instructor is desirable if it can be ob- 
tained without imposing on the instructors. Most of the pro- 
fessors are glad to help place their students. If the representative 
believes that the senior will be desired by the Company and that 
this senior will make application, he should get the rating from 
the professor while visiting the college or as soon as possible. 

3. As soon as the graduate student engineers enter the Edu- 
cational Department of the Company, they should be under ob- 
servation in the shop and in the class to be rated at the end of each 
month by means of the Shop and Class Rating Scale. 

4. Within the first two months after the graduate student 
engineer enters the Educational Department of the Company, he 
should be given the Test 10, A Test for Graduate Student En- 
gineers. This should be scored, first to determine for what kind 
of work the engineer is probably best fitted; second, to determine 
what the rank in general intelligence is. 

5. At the end of two months or just before it is necessary to 
segregate the graduate student engineers for intensive training in 
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some particular line of work, the student engineers should be 
requested to fill out the blank entitled Rceord of Interests. 

6. At the end of two months or when it is necessary to know 
into what line of work the student engineers are going, the re- 
sults of the rating scales, the test, and the Record of Interests, 
should be combined. The impressions and opinions concerning 
the engineers which the executives, interviewers, and instructors 
of the Educational Department have recorded cn the rating scales, 
should have a weight or vote of one-third in the final composite 
measure of the men; the test should have a vote or weight of one- 
third; and the Record of Interests should have a weight or vote 
of one-third to determine for what line of work the engineer 
should be recommended. In case of doubt, any two of these 
measures or criteria should determine for what work the partic- 
ular man should be recommended. Of course the judgments of 
the executives concerned will be the final criteria, but their im- 
pressions and opinions should be based upon dependable data and 
be corrected by this data. 

7. The student engineer should be interviewed with his 
Record of Interests before him and the interviewer; and in the 
interview the student should be advised to enter the line of work 
for which he is best fitted as determined by this Record of In- 
terests, the test, and the rating scales. This advice should be 
given as an intelligent recommendation; but the student should 
not be urged against his will to enter a particular line of work. 
If the student objects to the recommendation, it should be dis- 
cussed in the light of the results of the Record of Interests and of 
the test. 

8. After the student engineers have been segregated and as- 
signed tentatively to their respective lines of work, a copy of the 
Record of Interests and the results of the test should be trans- 
mitted to the heads of the respective departments in which the 
men will work. 

9g. The results of all rating scales, tests, and Record of In- 
terests, should be carefully recorded and filed; and the later suc- 
cess or record of the engineers should be systematically followed 
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up in order to check the reliability and value of the methods and 
means of selection, classification, and placement being used. 

10. Some person qualified for personnel research work should 
be assigned the responsibility for keeping rating scales and tests 
in proper use, working up the results, carrying on the research 
and follow-up work, and planning and directing new lines of 
research which should be undertaken to increase the efficiency of 
the educational, placement, and other personnel work. 

11. The rating scales, tests, and Record of Interests should be 
studied to make further improvements as a result of their use. A 
duplicate or alternate form of the test should be prepared, which 
could be given in case there is reason to believe there has been 
coaching for the first form. 

12. In the light of results obtained in the study of this prob- 
lem with graduate student engineers, research work in personnel 
problems should be extended to the apprentices, the clerical work- 
ers and the shop employees of the Company. 
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